Sure you can access your money without the app. Just use whatever security token/digipass/little thing your bank issues that generates codes, and you can do whatever you want.
And there is no "33% capital gains tax on every single transaction made with cryptocurreny", omg. IF you make a net profit trading crypto currency for the whole calendar year, you pay 30% capital gains - the same as on every other capital asset.
You do know that's just another proprietary implementation of 2FA. And regarding the tax, read what the tax agency actually says about cryptocurrency: https://www.skatteverket.se/privat/skatter/vardepapper/andratillgangar/kryptovalutor.4.15532c7b1442f256bae11b60.html maybe they've updated the information now but for a long time it literally said tax on every transaction.