HTML Navigation in C#

Using the Aspose.HTML for .NET library, you can easily create your own application, since our API provides a powerful toolset to analyze and collect information from HTML documents.

This article provides information on how to programmatically extract data from HTML documents with the Aspose.HTML for .NET API. You find out:

how to navigate through an HTML document and perform a detailed inspection of its elements using the API;
about custom filters usage for iterating over the document elements;
how to navigate over the document by using CSS Selector or XPath Query.

The Aspose.Html.Dom namespace provides API that represents and interacts with any HTML, XML or SVG documents and is entirely based on the WHATWG DOM specification supported in many modern browsers. The DOM is a document model loaded in the browser and representing the document as a node tree, where each node represents part of the document (e.g. an element, text string, or comment).

We consider how the DOM represents an HTML document in memory and how to use API for navigation through HTML files. Many ways can be used to make HTML navigation. The following shortlist shows the simplest way to access all DOM elements:

Property	Description
FirstChild	Accessing this property of an element must return a reference to the first child node.
LastChild	Accessing this property of an element must return a reference to the last child node
NextSibling	Accessing this property of an element must return a reference to the sibling node of that element which most immediately follows that element.
PreviousSibling	Accessing this property of an element must return a reference to the sibling node of that element which most immediately precedes that element.
ChildNodes	Returns a list that contains all children of that element.

Four of the Node class properties – FirstChild, LastChild, NextSibling, and PreviousSibling, each provides a live reference to another element with the defined relationship to the current element if the related element exists. For a complete list of classes and methods represented in the Aspose.Html.Dom Namespace, please visit API Reference Source.

Using the mentioned properties, you can navigate through an HTML document as it follows:

 1// Prepare HTML code
 2var html_code = "<span>Hello,</span> <span>World!</span>";
 3
 4// Initialize a document from the prepared code
 5using (var document = new HTMLDocument(html_code, "."))
 6{
 7    // Get the reference to the first child (first <span>) of the <body>
 8    var element = document.Body.FirstChild;
 9    Console.WriteLine(element.TextContent); // output: Hello,
10
11    // Get the reference to the whitespace between html elements
12    element = element.NextSibling;
13    Console.WriteLine(element.TextContent); // output: ' '
14
15    // Get the reference to the second <span> element
16    element = element.NextSibling;
17    Console.WriteLine(element.TextContent); // output: World!
18
19    // Set an html variable for the document 
20    var html = document.DocumentElement.OuterHTML;
21
22    Console.WriteLine(html); // output: <html><head></head><body><span>Hello,</span> <span>World!</span></body></html>
23}

view raw Example-NavigateThroughHTML hosted with ❤ by GitHub

Inspect HTML

Aspose.HTML contains a list of methods that are based on the Element Traversal Specifications. You can perform a detailed inspection of the document and its elements using the API. The following code sample shows the generalized usage of Element Traversal features.

 1// Load a document from a file
 2string documentPath = Path.Combine(DataDir, "html_file.html");
 3
 4using (var document = new HTMLDocument(documentPath))
 5{
 6    // Get the html element of the document
 7    var element = document.DocumentElement;                
 8    Console.WriteLine(element.TagName); // HTML
 9
10    // Get the last element of the html element
11    element = element.LastElementChild;
12    Console.WriteLine(element.TagName); // BODY
13
14    // Get the first element in the body element
15    element = element.FirstElementChild;
16    Console.WriteLine(element.TagName); // H1
17    Console.WriteLine(element.TextContent); // Header 1
18}

view raw Example-InspectHtmlDocument hosted with ❤ by GitHub

Note: You need to specify the path to the source HTML file in your local file system (documentPath).

The DocumentElement property of the Document class gives direct access to the <html> element of the document ( html_file.html). The LastElementChild property of the Document class returns the last child element of the <html> element. It is the <body> element. According to the code snippet above, the variable element is overloaded again, and the FirstElementChild property returns the first child of the <body> element. It is the <h1> element.

Custom Filter Usage

For the more complicated scenarios, when you need to find a node based on a specific pattern (e.g., get the list of headers, links, etc.), you can use a specialized TreeWalker or NodeIterator object with a custom Filter implementation.

The following example shows how to implement your own NodeFilter to skip all elements except images:

 1class OnlyImageFilter : Aspose.Html.Dom.Traversal.Filters.NodeFilter
 2{
 3    public override short AcceptNode(Node n)
 4    {
 5        // The current filter skips all elements, except IMG elements
 6        return string.Equals("img", n.LocalName)
 7            ? FILTER_ACCEPT
 8            : FILTER_SKIP;
 9    }
10}

view raw Example-OnlyImageFilter hosted with ❤ by GitHub

Once you implement a filter, you can use HTML navigation as it follows:

 1// Prepare HTML code
 2var code = @"
 3    <p>Hello,</p>
 4    <img src='image1.png'>
 5    <img src='image2.png'>
 6    <p>World!</p>";
 7
 8// Initialize a document based on the prepared code
 9using (var document = new HTMLDocument(code, "."))
10{
11    // To start HTML navigation, we need to create an instance of TreeWalker
12    // The specified parameters mean that it starts walking from the root of the document, iterating all nodes and using our custom implementation of the filter
13    using (var iterator = document.CreateTreeWalker(document, NodeFilter.SHOW_ALL, new OnlyImageFilter()))
14    {                    
15        while (iterator.NextNode() != null)
16        {
17            // Since we are using our own filter, the current node will always be an instance of the HTMLImageElement
18            // So, we don't need the additional validations here
19            var image = (HTMLImageElement)iterator.CurrentNode;
20
21            Console.WriteLine(image.Src);
22            // output: image1.png
23            // output: image2.png
24
25            // Set an html variable for the document 
26            var html = document.DocumentElement.OuterHTML;                        
27        }
28    }
29}

view raw Example-NodeFilterUsage hosted with ❤ by GitHub

XPath Query

The alternative to the HTML Navigation is XPath Query ( XML Path Language) that often referred to simply as an XPath. It is a query language that can be used to query data from HTML documents. It is based on a DOM representation of the HTML document, and selects nodes by various criteria. The syntax of the XPath expressions is quite simple, and what is more important, it is easy to read and support.

The following example shows how to use XPath queries within Aspose.HTML API:

 1// Prepare HTML code
 2var code = @"
 3    <div class='happy'>
 4        <div>
 5            <span>Hello,</span>
 6        </div>
 7    </div>
 8    <p class='happy'>
 9        <span>World!</span>
10    </p>
11";
12
13// Initialize a document based on the prepared code
14using (var document = new HTMLDocument(code, "."))
15{
16    // Here we evaluate the XPath expression where we select all child <span> elements from elements whose 'class' attribute equals to 'happy':
17    var result = document.Evaluate("//*[@class='happy']//span",
18        document,
19        null,
20        XPathResultType.Any,
21        null);
22
23    // Iterate over the resulted nodes
24    for (Node node; (node = result.IterateNext()) != null;)
25    {
26        Console.WriteLine(node.TextContent);
27        // output: Hello,
28        // output: World!
29    }
30}

view raw Example-XPathQueryUsage hosted with ❤ by GitHub

The article How to use XPath Query in HTML – Evaluate() method introduces how to use Evaluate() method to navigate through an HTML document and select nodes by various criteria. With C# examples, you will learn how to select all Nodes with specified Name using XPath query.

CSS Selector

Along with HTML Navigation and XPath, you can use CSS Selector API that is also supported by our library. This API is designed to create a search pattern to match elements in a document tree based on CSS Selectors syntax.

In the following example, we use the QuerySelectorAll() method for navigation through an HTML document and search the needed elements. This method takes as a parameter the query selector and returns a NodeList of all the elements, which match the specified selector.

 1// Prepare HTML code
 2var code = @"
 3    <div class='happy'>
 4        <div>
 5            <span>Hello,</span>
 6        </div>
 7    </div>
 8    <p class='happy'>
 9        <span>World!</span>
10    </p>
11";
12
13// Initialize a document based on the prepared code
14using (var document = new HTMLDocument(code, "."))
15{
16    // Here we create a CSS Selector that extracts all elements whose 'class' attribute equals 'happy' and their child <span> elements
17    var elements = document.QuerySelectorAll(".happy span");
18
19    // Iterate over the resulted list of elements
20    foreach (HTMLElement element in elements)
21    {
22        Console.WriteLine(element.InnerHTML);
23        // output: Hello,
24        // output: World!
25    }
26}

view raw Example-CSSSelectorUsage hosted with ❤ by GitHub

See Also

For more information on how to effectively apply selectors to select the elements you want to style, please visit the article CSS Selectors. You will cover Basic Selectors, Combinator Selectors, Attribute Selectors, Group Selectors and Pseudo Selectors.
In the article How to use CSS Selector – QuerySelector() and QuerySelectorAll(), you will learn how to effectively apply selectors to select the elements you want to style. QuerySelector() and QuerySelectorAll() are methods that are used to query DOM elements that match a CSS selector.
You can download the complete C# examples and data files from GitHub.
Aspose.HTML offers free online HTML Web Applications that are an online collection of converters, mergers, SEO tools, HTML code generators, URL tools, and more. The applications work on any operating system with a web browser and do not require any additional software installation. Easily convert, merge, encode, generate HTML code, extract data from the web, or analyze web pages in terms of SEO wherever you are. Use our collection of HTML Web Applications to perform your daily matters and make your workflow seamless!

Data Extraction Website to HTML