How can I select rows with most recent timestamp for each key value?
For the sake of completeness, here's another possible solution:
SELECT sensorID,timestamp,sensorField1,sensorField2
FROM sensorTable s1
WHERE timestamp = (SELECT MAX(timestamp) FROM sensorTable s2 WHERE s1.sensorID = s2.sensorID)
ORDER BY sensorID, timestamp;
Pretty self-explaining I think, but here's more info if you wish, as well as other examples. It's from the MySQL manual, but above query works with every RDBMS (implementing the sql'92 standard).
In Postgres this can de done in a relatively elegant way using SELECT DISTINCT
, as follows:
SELECT DISTINCT ON (sensorID)
sensorID, timestamp, sensorField1, sensorField2
FROM sensorTable
ORDER BY sensorID, timestamp DESC;
Some more info here. I suspect it also works for other SQL flavors, though apparently not MySQL (link - thanks for the tip @silentsurfer)
In case it's not obvious, what this does is sort the table by sensor ID and timestamp (newest to oldest), and then returns the first row (i.e. latest timestamp) for each unique sensor ID.
In my use case I have ~10M readings from ~1K sensors, so trying to join the table with itself on a timestamp-based filter is very resource-intensive; the above takes a couple of seconds.
You can only select columns that are in the group or used in an aggregate function. You can use a join to get this working
select s1.*
from sensorTable s1
inner join
(
SELECT sensorID, max(timestamp) as mts
FROM sensorTable
GROUP BY sensorID
) s2 on s2.sensorID = s1.sensorID and s1.timestamp = s2.mts