Looking for a cheap (I'm talking about cheap as in Thailand) hostel is almost impossible. There are no hostels, only guest houses or hotels (don't stay at a fancy hotel or do group tours, you will just support the government not the locals. Travel individually!!). I only know of the Okinawa Guest House in Chinatown that charges 8 US$ per night for the cheapest room. Or if you want to save your money and get to learn local people you should try out Couchsurfing. YES, Couchsurfing is not only possible in cities like Berlin or Barcelona, there are also Couchsurfers in Myanmar, believe it or not. So that's what I did and it was the best idea ever. Living in downtown where all the accommodations and touristic sights are is very convinient, I agree with that. But living outside of downtown in a local area can be so much more interesting and people are more curious to make friends with you and want to know why the hell there's a foreigner in their district. Don't be shy and just talk and make friends. Burmese people are so friendly and open and they do speak a bit english, so communication is not a problem. Add them on Facebook, exchanged e-mail addresses and you've made their day.
I stayed in Yangon for two days which was just right. I'm not really into big cities especially not at that moment due to three full days in Kuala Lumpur before Yangon.
Visit the Shwedagon temple, walk around downtown, spend some time in a Burmese tea house and have a drink in the northern part of the city when evening comes.