HtmlAgilityPack and HtmlDecode

I am currently using HtmlAgilityPack with a console application to scrape a website. Since the html is encoded (it returns encoded characters like ') I have to decode before I save the content to my database.

Is there a way to decode the returned html using HtmlAgilityPack without having to use HttpUtility.HtmlDecode? I want to avoid adding System.Web to my console application if possible.

Solution 1:

The Html Agility Pack is equiped with a utility class called HtmlEntity. It has a static method with the following signature:

/// <summary>
/// Replace known entities by characters.
/// </summary>
/// <param name="text">The source text.</param>
/// <returns>The result text.</returns>
public static string DeEntitize(string text)

It supports well-known entities (like  ) and encoded characters such as ' as well.

Solution 2:

Just adding my 2 cents: I've ran some performance tests using StopWatch class and found that HttpUtility.HtmlDecode is about 15-20% faster than the DeEntitize method. Also DeEntitize has some bugs (see comments above).

So maybe referencing System.Web is not that bad after all.

If you're writing an app that already targets ".NET full" (opposed to ".NET Client Profile" - which is a lightweight version) - I'd go for referencing System.Web.

Solution 3:

Use WebUtility that doesn't need any special reference.