A couple of days ago I was dealing with a similar scenario, a web application that must be deployed to Azure targeting the .Net framework 4.0 and using WIF as the authentication mechanism.
I remembered a tweet by Vittorio Bertocci:
And I realized that the source of the problem is not in the multi-targeting (that has is drawbacks) but is basically due to the new Identity and Access Visual Studio extension that does not take into account that the application that we are configuring does not target the .Net framework 4.5.
The sneaky problem is that on a machine with .Net framework 4.5 installed you cannot realize that unless in the application you try to access the underlying claim identity.
The easiest way to fix the problem is to have the WIF SDK installed on the machine and use Visual Studio 2010 and the Federation Utility just for the purpose of creating the WIF configuration the first time, than we can manually adjust it when requirements change, and do the rest of the development using Visual Studio 2012 as usual.
Another possible workaround is to use Web Config Transformation to change the WIF configuration at web application publish time, in this case we have to deal with more complexity but we have the advantage that our application can run both on .Net framework 4.0 with WIF SDK installed and even on .Net framework 4.5 in an environment where we cannot install anything other than the framework itself (typically an shared hosting environment).