Costs so much though. Thinking about what a phone even needs to do for me, what other handheld devices could work instead?

Steamdeck perhaps? Then a brick phone for calls/sms. Not too sure what would work as a lower spec and cheaper alternative to a Steamdeck and runs Linux though.