(Reuters) - Australia's biggest banks are likely to report weaker first-half profit as high operating costs and competition to sell mortgages and deposits squeeze margins, setting up a possible reversal of a stock rally that analysts said has left the sector overheated.

Traditionally beneficiaries of rising interest rates, the country's so-called Big Four lenders have instead spent the past year sacrificing margins to write new home loans and paying more to depositors, narrowing their closely watched "net interest margin" and putting downward pressure on profit, analysts said.

That may undermine a share price run-up in the sector since late 2023 that was based on signs that 13 interest rate hikes had tamed inflation and hopes of a return to rate cuts in 2024. Some analysts now expect a longer wait for rate cuts, with some suggesting the next move could be upwards.

"We expect further margin erosion in the first half of fiscal 2024, as the sector continues to be impacted by deposit/mortgage competition and adverse deposit mix shifts," analysts at investment and advisory firm Jarden wrote in a client note.

National Australia Bank ( NAUBF ) , the second-biggest mortgage lender and biggest business lender, is set to report a nine-basis-point narrowing in its first-half net interest margin when it begins the reporting season on May 2, with cash likely falling as much as 13%, analysts estimated.

"As the economy slows at some juncture, we expect that NAB as the largest business bank will be at a structural disadvantage versus peers," wrote Citi analysts, who recently downgraded their recommendation on Big Four shares to "sell".

"As business credit slows and becomes more competitive, relative momentum will disappoint and put pressure on (NAB's) expanded premium valuation."

Similar decline in margin and underlying profit is expected for Westpac Banking ( WEBNF ) and ANZ Group ( ANZGF ), according to market data aggregator Visible Alpha and other brokerages. Westpac and ANZ, the third- and fourth-biggest banks, announce half-year earnings on May 6 and 7 respectively.

Commonwealth Bank of Australia ( CBAUF ), Australia's biggest lender, gives a third-quarter trading update on May 9, with analysts expecting margin decline of as much as 11 basis points and profit decline of as much as 10%.

The lenders' shares have risen around one-fifth since October when investors began forecasting that economic data indicating inflation was being tamed would prompt central banks to start cutting rates from mid-2024.

However, as unfavourable economic data poured in from the start of the year, bets were dialled down, with total easing expectation now at 3 basis points, and even a small chance of a rate hike being now priced in. [0#RBAWATCH]

"This pause in rates would leave the bank rally exposed, with little reconciliation between multiples and the dour earnings outlook currently in consensus," Citi analysts said.