What you must be having is a fracture of the fifth metacarpal bone.Very common when you punch a wall in anger
Normally a 'z' plaster is applied or a brace to keep the fracture stable and allow healing to occur.
Callus formation or in lay man's words,the cementing of the fractured bones take about 2 weeks and then it needs to become strong enough for a few more weeks then the brace or plaster can be removed.
After 2 weeks you can remove the plaster/brace because the bone is not strong enough to withstand another impact or trauma,as it will break again!!!
To be on the safe side a good 4-6 weeks in needed.If there was been good healing then after 4 weeks the plaster/brace can be removed early physiotherapy started to regain the loss of function due to immobilization.A course of antiinflammatory/painkiller drugs is necessary to decrease the swelling and pain as well.
Hope this helps.