Pretty straight forward but here's nice visual to go along.
http://img.photobucket.com/albums/v202/JossiRossi/Hammerissue.png

I did try to find the answer myself both here on the forums and on google but I wasn't finding anything (probably because the search things I could come up with were too generic and anything more specific gave me nothing). But I can't figure out why it does this, and when I try to click on one option or the other nothing happens, they both stay clicked. If anyone has any ideas I'd appreciate it.

you can select the whole brush (not just the problematic face) then click align to face then align to world, if they are both stil checked then you will most likely haveto recreate the brush and make sure not to select align to world first, this has been a bug in hammer since it was worldcraft

I've been getting this problem quite consistently in Hammer 4.1. It was present in older versions, but I get it now even when I simply create a brush. It's like a configuration file has to be reset or something.