When considering street photography, maybe we should make a difference between what is legally possible, what is polite practice and what risks there are.
When an amateur asks what he or she is allowed to do with portraits, the question actually means: "what can I do without risking to be sued?". The answer is quite simply: "nothing". Going to court is a right and anyone is allowed to sue you for whatever reason they see fit. They may lose, the judge may even dismiss the complain prima facie as frivolous, but the bottom line is that anyone
can sue because you simply have a camera.
The next question is then "what are the risk of losing a lawsuit?". That is a quite different question and the answer depends on what countries are involved, jurisprudence, particulars of the situation and who the photographer is. Moreover, the jurisprudence evolves everyday. For example: views of crowds used to be relatively safe. With increase in pixel counts and availability of mass face recognition software, I am not so sure that they will be in a few years.
As to what increases the probability of losing a lawsuit, I would say that the main reasons probably are:
- publishing the picture (for unpublished pictures no damages can be claimed)
- making profit (as damages can be better estimated)
- being an amateur photographer (as there are better protection for journalists and artists, and: no, you cannot convince a court that you are an artist if it is not your source of income)
- that the photographer could acertain that the model did not want an image to be taken or hid his/her intentions (i.e. the absence of informed consent)
- motives being linked to sexual desire (that part is interesting, because it is not actually in the law, but courts tend to follow the general public opinion and it has become more restrictive in that respect than in the 70s, for example)
- that the image is detrimental to the model, mocking the model for example.
The third question was about "polite practice", for the lack of a better term. That is also a different question. Some people, for example the homeless, people in third world countries, victims of major disasters, are unlikely to sue because they simply lack the ressources. Does the absence of legal risk mean they are free game? I think not. Yet social networks thrive on such images.