Facebook FQL stream limit?

I want to get the full history of my wall. But I seem to hit a limit somewhere back in June.

I do multiple calls like this:

SELECT created_time,message FROM stream WHERE source_id=MY_USER_ID LIMIT 50

SELECT created_time,message FROM stream WHERE source_id=MY_USER_ID LIMIT 51,100

and so on...

But I always end up on the same last (first) post on my wall. Through facebook.com I can go back much longer so Facebook obviously have the data.

Why am I not getting older posts? Is there another way to scrape my history?

From http://developers.facebook.com/docs/reference/fql/stream :

The stream table is limited to the last 30 days or 50 posts, whichever is greater

I am experiencing the same thing. I don't understand it at all, but it appears that the offset cannot be greater than the limit * 1.5

Theoretically, this means that always increasing the limit to match the offset would fix it, but I haven't been able to verify this (I'm not sure whether the problems I'm seeing are other bugs in my code or if there are other limitations I don't understand about getting the stream).

Can anyone explain what I'm seeing and whatever I'm missing?

You can reproduce my results by going to the FQL Test Console:

http://developers.facebook.com/docs/reference/rest/fql.query

pasting in this query:

SELECT post_id, created_time, message, likes, comments, attachment, permalink, source_id, actor_id 
FROM stream 
WHERE filter_key IN 
(
      SELECT filter_key 
      FROM stream_filter 
      WHERE uid=me() AND type='newsfeed'
) 
AND is_hidden = 0 limit 100 offset 150

When you click "Test Method" you will see one of the 2 results I am getting:

1. The results come back: [{post_id:"926... (which I expected)
2. It returns empty [] (which I didn't expect)

You will likely need to experiment by changing the "offset" value until you find the exact place where it breaks. Just now I found it breaks for me at 155 and 156.

Try changing both the limit and the offset and you'll see that the empty results don't occur at a particular location in the stream. Here are some examples of results I've seen:

• "...limit 50 offset 100" breaks, returning empty []
• "...limit 100 offset 50" works, returning expected results
• "...limit 50 offset 74" works
• "...limit 50 offset 75" breaks
• "...limit 20 offset 29" works
• "...limit 20 offset 30" breaks

Besides seeing the limit=offset*1.5 relationship, I really don't understand what is going on here.

Skip the FQL and go straight to graph. I tried FQL and it was buggy when it came to limits and getting specified date ranges. Here's the graph address. Put in your own page facebook_id and access_token:

https://graph.facebook.com/FACEBOOK_ID/posts?access_token=ACCESS_TOKEN

Then if you want to get your history set your date range using since, until and limit:

https://graph.facebook.com/FACEBOOK_ID/posts?access_token=ACCESS_TOKEN&since=START_DATE&until=END_DATE&limit=1000

Those start and end dates are in unix time, and I used limit because if I didn't it would only give me 25 at a time. Finally if you want insights for your posts, you'll have to go to each individual post and grab the insights for that post:

https://graph.facebook.com/POST_ID/insights?access_token=ACCESS_TOKEN