I never heard of Astoria before. I did hear about ADO.NET DataService, but as there's ADO in the name it had to be evil. Luckily Astoria is not about ADO.NET, it's about exposing data using JSON & REST.
Basically the idea is very simple. You create a website project, add a class that will act as your data source. This can be anything, be it Linq to Sql, Entity or NHibernate. You can even use a stub to temporarily make some data available. The only important thing to keep in mind is that it needs to implement IQueryable for select operations.
Then add a new item - ADO.NET Data Services item.
Go to the newly created cs file and implement the DataServices generic base class.
That's all there is to it. You can now reference the WCF Data Service and query it like you would query any IEnumerable<T>.
Very cool stuff.
Some links for Astoria:
Team blog Astoria Learning Astoria MSDN Reference API Documentation Help forum and info Whitepaper on using Astoria
Watch the screencast here.