Whats your budget? HK is expensive. Even dangerous crappy bug infested places will run around 400 now. But if you bump it up to lile the 700 range you can have a decent sleep without worry of getting robbed and or infested. To save a few bucks, weekdays are much cheaper than weekends usually.

I stayed in a very nice dorm at YHA Mei Ho House for 260 a night, that's YHA membership price including a nice breakfast; reg. price is 360. It's a converted old gov't housing project but the bed was comfortable and everything well-maintained.

Go to Booking and look for what is near Wan Chai 湾仔. It's not the cheapest place in HK but it's not overly expensive either, and near the visa office. Assuming you go on a weekday, you should find decent 3 stars 15m² rooms from 400 to 800 a night.

If you are really on a tight budget, You can chose the infamous Tsim Sha Tsui, 尖沙咀, just two stations or a boat away from the visa place, but the hostels there are... let's say it's the lottery.

http://www.hostelbookers.com/results/loc/dd/dst/2954/arr/2014-06-11/ngt/...

Embassy is in Wanchai. You can cross from Mong Kok (budget) to Wanchai by boat for HK$ 2.5 one way (30').