A colleague of mine just installed the new iPhone Flickr app. He then added me as his contact. As soon as he did, he could see all my private photos before I added him as a contact! I believe this is a major security flaw, that gives anyone with an iPhone access to photos that are not publically shared.

Update: I reapplied the security settings on all of my photos, to make sure that they are correct. This seems to have solved the issue partly. I still do not know why photos were visable in the iPhone app and not my photo stream online.