Yahoo Finance URL not working

yahoo-finance

I have been using the following URL to fetch historical data from yahoo finance for quite some time now but it stopped working as of yesterday.

https://ichart.finance.yahoo.com/table.csv?s=SPY

When browsing to this site it says:

Will be right back...

Thank you for your patience.

Our engineers are working quickly to resolve the issue.

However, since this issue is still existing since yesterday I am starting to think that they discontinued this service?

My SO search only pointed me to this topic, which was related to https though...

Is anyone else experiencing this issue? How can I resolve this problem? Do they offer a different access to their historical data?

Yahoo has gone to a Reactjs front end which means if you analyze the request headers from the client to the backend you can get the actual JSON they use to populate the client side stores.

Hosts:

query1.finance.yahoo.com HTTP/1.0
query2.finance.yahoo.com HTTP/1.1 (difference between HTTP/1.0 & HTTP/1.1)

If you plan to use a proxy or persistent connections use query2.finance.yahoo.com. But for the purposes of this post, the host used for the example URLs is not meant to imply anything about the path it's being used with.

Fundamental Data

(substitute your symbol for: AAPL)

/v10/finance/quoteSummary/AAPL?modules=

Inputs for the ?modules= query:

    [
       'assetProfile',
       'summaryProfile',
       'summaryDetail',
       'esgScores',
       'price',
       'incomeStatementHistory',
       'incomeStatementHistoryQuarterly',
       'balanceSheetHistory',
       'balanceSheetHistoryQuarterly',
       'cashflowStatementHistory',
       'cashflowStatementHistoryQuarterly',
       'defaultKeyStatistics',
       'financialData',
       'calendarEvents',
       'secFilings',
       'recommendationTrend',
       'upgradeDowngradeHistory',
       'institutionOwnership',
       'fundOwnership',
       'majorDirectHolders',
       'majorHoldersBreakdown',
       'insiderTransactions',
       'insiderHolders',
       'netSharePurchaseActivity',
       'earnings',
       'earningsHistory',
       'earningsTrend',
       'industryTrend',
       'indexTrend',
       'sectorTrend']

Example URL: querying for all of the above modules

https://query2.finance.yahoo.com/v10/finance/quoteSummary/AAPL?modules=assetProfile%2CsummaryProfile%2CsummaryDetail%2CesgScores%2Cprice%2CincomeStatementHistory%2CincomeStatementHistoryQuarterly%2CbalanceSheetHistory%2CbalanceSheetHistoryQuarterly%2CcashflowStatementHistory%2CcashflowStatementHistoryQuarterly%2CdefaultKeyStatistics%2CfinancialData%2CcalendarEvents%2CsecFilings%2CrecommendationTrend%2CupgradeDowngradeHistory%2CinstitutionOwnership%2CfundOwnership%2CmajorDirectHolders%2CmajorHoldersBreakdown%2CinsiderTransactions%2CinsiderHolders%2CnetSharePurchaseActivity%2Cearnings%2CearningsHistory%2CearningsTrend%2CindustryTrend%2CindexTrend%2CsectorTrend

The %2C is the Hex representation of , and needs to be inserted between each module you request. details about the hex encoding bit(if you care)

Options contracts

/v7/finance/options/AAPL (current expiration)
/v7/finance/options/AAPL?date=1679011200 (March 17, 2023 expiration)

Example URL:

https://query2.finance.yahoo.com/v7/finance/options/AAPL (current expiration)
https://query2.finance.yahoo.com/v7/finance/options/AAPL?date=1679011200 (Match 17, 2023 expiration)

Any valid future expiration represented as a UNIX timestamp can be used in the ?date= query. If you query for the current expiration the JSON response will contain a list of all the valid expirations that can be used in the ?date= query. (here is a post explaining converting human-readable dates to UNIX timestamp in Python)

Price

/v8/finance/chart/AAPL?symbol=AAPL&period1=0&period2=9999999999&interval=3mo

Possible inputs for &interval=: 1m, 5m, 15m, 30m, 90m, 1h, 1d, 5d, 1wk, 1mo, 3mo

m (minute) intervals are limited to 30days with period1 and period2 spaning a maximum of 7 days per/request. Exceeding either of these limits will result in an error and will not round

h (hour) interval is limited to 730days with no limit to span. Exceeding this will result in an error and will not round

period1=: UNIX timestamp representation of the date you wish to start at.

d (day), wk (week), mo (month) intervals with values less than the initial trading date will be rounded up to the initial trading date.

period2=: UNIX timestamp representation of the date you wish to end at.

For all intervals: values greater than the last trading date will be rounded down to the most recent timestamp available.

Add pre & post market data

&includePrePost=true

Add dividends & splits

&events=div%7Csplit

%7C is hex for |. , will work but internally yahoo uses pipe

Example URL:

https://query1.finance.yahoo.com/v8/finance/chart/AAPL?symbol=AAPL&period1=0&period2=9999999999&interval=1d&includePrePost=true&events=div%7Csplit

The above request will return all price data for ticker AAPL on a 1-day interval including pre and post-market data as well as dividends and splits.

Note: the values used in the price example URL for period1= & period2= are to demonstrate the respective rounding behavior of each input.`

It looks like they have started adding a required cookie, but you can retrieve this fairly easily, for example:

GET https://uk.finance.yahoo.com/quote/AAPL/history

Responds with the header in the form:

set-cookie:B=xxxxxxxx&b=3&s=qf; expires=Fri, 18-May-2018 00:00:00 GMT; path=/; domain=.yahoo.com

You should be able to read this and attach it to your .csv request:

GET https://query1.finance.yahoo.com/v7/finance/download/AAPL?period1=1492524105&period2=1495116105&interval=1d&events=history&crumb=tO1hNZoUQeQ
cookie: B=xxxxxxxx&b=3&s=qf;

Note the crumb query parameter, this seems to correspond to your cookie in some way. Your best bet is to scrape this from the HTML response to your initial GET request. Within that response, you can do a regex search for: "CrumbStore":\{"crumb":"(?<crumb>[^"]+)"\} and extract the crumb matched group.

It looks like once you have that crumb value though you can use it with the same cookie on any symbol/ticker for the next year meaning you shouldn't have to do the scrape too frequently.

To get current quotes just load:

https://query1.finance.yahoo.com/v8/finance/chart/AAPL?interval=2m

With:

AAPL substituted with your stock ticker
interval one of [1m, 2m, 5m, 15m, 30m, 60m, 90m, 1h, 1d, 5d, 1wk, 1mo, 3mo]
optional period1 query param with your epoch range start date e.g. period1=1510340760
optional period2 query param with your epoch range end date e.g. period2=1510663712

I managed to work out a .NET class to obtain valid token (cookie and crumb) from Yahoo Finance

For complete API library in fetching historical data from new Yahoo Finance, you may visit YahooFinanceAPI in Github

Here is the class to grab the cookie and crumb

Token.cs

using System;
using System.Diagnostics;
using System.Net;
using System.IO;
using System.Text.RegularExpressions;

namespace YahooFinanceAPI
{
    /// <summary>
    /// Class for fetching token (cookie and crumb) from Yahoo Finance
    /// Copyright Dennis Lee
    /// 19 May 2017
    /// 
    /// </summary>
    public class Token
    {
        public static string Cookie { get; set; }
        public static string Crumb { get; set; }

        private static Regex regex_crumb;
        /// <summary>
        /// Refresh cookie and crumb value Yahoo Fianance
        /// </summary>
        /// <param name="symbol">Stock ticker symbol</param>
        /// <returns></returns>
        public static bool Refresh(string symbol = "SPY")
        {

            try
            {
                Token.Cookie = "";
                Token.Crumb = "";

                string url_scrape = "https://finance.yahoo.com/quote/{0}?p={0}";
                //url_scrape = "https://finance.yahoo.com/quote/{0}/history"

                string url = string.Format(url_scrape, symbol);

                HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);

                request.CookieContainer = new CookieContainer();
                request.Method = "GET";

                using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
                {

                    string cookie = response.GetResponseHeader("Set-Cookie").Split(';')[0];

                    string html = "";

                    using (Stream stream = response.GetResponseStream())
                    {
                        html = new StreamReader(stream).ReadToEnd();
                    }

                    if (html.Length < 5000)
                        return false;
                    string crumb = getCrumb(html);
                    html = "";

                    if (crumb != null)
                    {
                        Token.Cookie = cookie;
                        Token.Crumb = crumb;
                        Debug.Print("Crumb: '{0}', Cookie: '{1}'", crumb, cookie);
                        return true;
                    }

                }

            }
            catch (Exception ex)
            {
                Debug.Print(ex.Message);
            }

            return false;

        }

        /// <summary>
        /// Get crumb value from HTML
        /// </summary>
        /// <param name="html">HTML code</param>
        /// <returns></returns>
        private static string getCrumb(string html)
        {

            string crumb = null;

            try
            {
                //initialize on first time use
                if (regex_crumb == null)
                    regex_crumb = new Regex("CrumbStore\":{\"crumb\":\"(?<crumb>.+?)\"}", 
                RegexOptions.CultureInvariant | RegexOptions.Compiled, TimeSpan.FromSeconds(5));

                MatchCollection matches = regex_crumb.Matches(html);

                if (matches.Count > 0)
                {
                    crumb = matches[0].Groups["crumb"].Value;
                }
                else
                {
                    Debug.Print("Regex no match");
                }

                //prevent regex memory leak
                matches = null;

            }
            catch (Exception ex)
            {
                Debug.Print(ex.Message);
            }

            GC.Collect();
            return crumb;

        }

    }
}

Updated 1 Jun 17
credits to @Ed0906
modify crumb regex pattern to Regex("CrumbStore\":{\"crumb\":\"(?<crumb>.+?)\"}"

Yahoo Finance URL not working

Hosts:

Fundamental Data

Options contracts

Price

Related

Recent Posts