Browsing web pages
Let's start with basic web integration patterns. The first thing we will do is to do a simple navigation through several pages to introduce the way of working with ITPilot. For this, we will navigate to the Yahoo news page and access their section for World News.
Create a new wrapper named "yahoonews":
As the wrapper is going to be browsing the web, the first step is to add a Sequence component to the wrapper. Drag it from the left component tree onto the workspace.
Let's link the Sequence component to the other components, like this:
Init -> Sequence_1 -> End.The Output component is not needed so click on it and delete it by pressing the "Delete" key.
You will have noticed that the Sequence component is displayed in red. That's because it's not configured yet, so let's tell ITPilot how to navigate to the pages we want to visit. We are going to do this by recording our actions directly in the browser. Open a new browser from within the WGT by clicking the menu option
Browser > New browser.
You will need to open the MSIE browser from within the Denodo Platform. If the browser is started from the regular Windows menu, it will not be able to communicate with ITPilot. Always make sure your browsers are started by ITPilot.
TIPYou can see that MSIE now has a browser toolbar for Denodo. Let's start the navigation to our first page by clicking the
Recbutton in the toolbar, for recording a navigation sequence. That displays a dialog that asks us what URL we want to use as our starting point for the navigation. Type http://news.yahoo.com/ and click "Ok".
The browser navigates normally to the page that we specified, and it is recording the sequence of actions that we perform on the pages. Let's now click a link and go to the "World News" section. If you click normally on the link you will realize that nothing is happening. That's because ITPilot is intercepting the mouse actions and stopping them; what you need to do to issue a click or any other action is to right click on the link: this will display a contextual menu with all the actions that we can do over the page. Select "Click" and you will see that our browser window indeed navigates to the next page.
Another thing we can do is search for a term in the page. You can see the Yahoo News search box at the top of the page, so right click over it and select
Set Text. A new dialog comes up and asks the text to type in the search field. Type "Technology" and click "Ok". The text should appear in the search box.
To proceed with the search, right click over the "Search News" button and select "Click". We will reach the search results page.
Our sequence of navigation actions is now complete. Click the
Stopbutton on the MSIE Denodo toolbar to complete the recording. Do not close the browser window!
Back in the WGT, double-click on the sequence_1 component to open its configuration wizard. You can see an
Import from browserbutton - click it to retrieve the navigation sequence that we just recorded in the browser into the Sequence wizard, and click "Ok" to accept the Sequence component configuration.
If you review the code that was imported into the Sequence component, you will probably not be familiar with the syntax. ITPilot expresses navigation sequences in NSEQL (Navigation SEQuences Language), which was created by Denodo to describe browsing at a high level.
Check the ITPilot documentation for a complete reference of all the commands available, and check our Denodo Platform training courses for a complete class in the use of NSEQL.NOTEOur wrapper is now complete. Type
CTRL+Sto save it.
Finally, let's execute the wrapper so we can see ITPilot doing the automatic navigation. Click on the
Test wrapperbutton at the bottom of your screen, and click "Execute wrapper" in the next dialog. This will start the wrapper execution, and you should see a new browser window appear, navigate to yahoo news, go to the world news section and search for "Technology" in the news.
This wrapper is now complete; it does not return any data but it is demonstrating how the first web automation steps work. In the following examples we will apply these basics to do more useful work.