Request for the permission of type 'System.Security.Permissions.SecurityPermission, mscorlib, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089' failed.
Not very helpful. So, after a bit of Googling I found this post, pointing towards a filelogger for the OpenID activity trace. As is suggested there, I commented out the entire system.diagnostics section from the web.config, since it only deals with this particular trace. After that, the site worked like a charm.
system.diagnostics