Get bitcoin historical data [closed]

Actually, you CAN get the whole Bitcoin trades history from Bitcoincharts in CSV format here : http://api.bitcoincharts.com/v1/csv/

it is updated twice a day for active exchanges, and there is a few dead exchanges, too.

EDIT: Since there are no column headers in the CSVs, here's what they are : column 1) the trade's timestamp, column 2) the price, column 3) the volume of the trade

You can find a lot of historical data here: https://www.quandl.com/data/BCHARTS-Bitcoin-Charts-Exchange-Rate-Data

In case, you would like to collect bitstamp trade data form their websocket in higher resolution over longer time period you could use script log_bitstamp_trades.py below.

The script uses python websocket-client and pusher_client_python libraries, so install them.

#!/usr/bin/python

import pusherclient
import time
import logging
import sys
import datetime
import signal
import os

logging.basicConfig()
log_file_fd = None

def sigint_and_sigterm_handler(signal, frame):
    global log_file_fd
    log_file_fd.close()
    sys.exit(0)


class BitstampLogger:

    def __init__(self, log_file_path, log_file_reload_path, pusher_key, channel, event):
        self.channel = channel
        self.event = event
        self.log_file_fd = open(log_file_path, "a")
        self.log_file_reload_path = log_file_reload_path
        self.pusher = pusherclient.Pusher(pusher_key)
        self.pusher.connection.logger.setLevel(logging.WARNING)
        self.pusher.connection.bind('pusher:connection_established', self.connect_handler)
        self.pusher.connect()

    def callback(self, data):
        utc_timestamp = time.mktime(datetime.datetime.utcnow().timetuple())
        line = str(utc_timestamp) + " " + data + "\n"
        if os.path.exists(self.log_file_reload_path):
            os.remove(self.log_file_reload_path)
            self.log_file_fd.close()
            self.log_file_fd = open(log_file_path, "a")
        self.log_file_fd.write(line)

    def connect_handler(self, data):
        channel = self.pusher.subscribe(self.channel)
        channel.bind(self.event, self.callback)


def main(log_file_path, log_file_reload_path):
    global log_file_fd
    bitstamp_logger = BitstampLogger(
        log_file_path,
        log_file_reload_path,
        "de504dc5763aeef9ff52",
        "live_trades",
        "trade")
    log_file_fd = bitstamp_logger.log_file_fd
    signal.signal(signal.SIGINT, sigint_and_sigterm_handler)
    signal.signal(signal.SIGTERM, sigint_and_sigterm_handler)
    while True:
        time.sleep(1)


if __name__ == '__main__':
    log_file_path = sys.argv[1]
    log_file_reload_path = sys.argv[2]
    main(log_file_path, log_file_reload_path

and logrotate file config

/mnt/data/bitstamp_logs/bitstamp-trade.log
{
    rotate 10000000000
    minsize 10M
    copytruncate
    missingok
    compress
    postrotate
        touch /mnt/data/bitstamp_logs/reload_log > /dev/null
    endscript
}

then you can run it on background

nohup ./log_bitstamp_trades.py /mnt/data/bitstamp_logs/bitstamp-trade.log /mnt/data/bitstamp_logs/reload_log &

Bitstamp has live bitcoin data that are publicly available in JSON at this link. Do not try to access it more than 600 times in ten minutes or else they'll block your IP (plus, it's unnecessary anyway; read more here). The below is a C# approach to getting live data:

using (var WebClient = new System.Net.WebClient())
{
     var json = WebClient.DownloadString("https://www.bitstamp.net/api/ticker/");
     string value = Convert.ToString(json);
     // Parse/use from here
}

From here, you can parse the JSON and store it in a database (or with MongoDB insert it directly) and then access it.

For historic data (depending on the database - if that's how you approach it), do an insert from a flat file, which most databases allow you to use (for instance, with SQL Server you can do a BULK INSERT from a CSV file).

I have written a java example for this case:

Use json.org library to retrieve JSONObjects and JSONArrays. The example below uses blockchain.info's data which can be obtained as JSONObject.

    public class main 
    {
        public static void main(String[] args) throws MalformedURLException, IOException
        {
            JSONObject data = getJSONfromURL("https://blockchain.info/charts/market-price?format=json");
            JSONArray data_array = data.getJSONArray("values");

            for (int i = 0; i < data_array.length(); i++)
            {
                JSONObject price_point = data_array.getJSONObject(i);

                //  Unix time
                int x = price_point.getInt("x");

                //  Bitcoin price at that time
                double y = price_point.getDouble("y");

                //  Do something with x and y.
            }

        }

        public static JSONObject getJSONfromURL(String URL)
        {
            try
            {
                URLConnection uc;
                URL url = new URL(URL);
                uc = url.openConnection();
                uc.setConnectTimeout(10000);
                uc.addRequestProperty("User-Agent", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)");
                uc.connect();

                BufferedReader rd = new BufferedReader(
                        new InputStreamReader(uc.getInputStream(), 
                        Charset.forName("UTF-8")));

                StringBuilder sb = new StringBuilder();
                int cp;
                while ((cp = rd.read()) != -1)
                {
                    sb.append((char)cp);
                }

                String jsonText = (sb.toString());            

                return new JSONObject(jsonText.toString());
            } catch (IOException ex)
            {
                return null;
            }
        }
    }