Last modified on: 07 May 2020
Twitter is a social networking site that relies on micro-blogging for communication. By typing short, 140 character messages (up to 280 since 2017) users can communicate with their followers and get information spread around. Since its inception, Twitter has grown and turned into a massive site used by mainstream media as well as average individuals to communicate about important issues, random issues, or just about what they’re doing at any time. It allows people to blog in short bursts, a practice also known as microblogging.
Twitter lets you make short posts about anything. These short posts are known as “tweets”, in reference to the chirping of birds, and appear chronologically on your Twitter profile page. In addition to posting, you can also follow others on Twitter to get their latest tweets in your Twitter feed. Conversely, if someone follows you, they get your tweets in their feeds.
Importing the Denodo Templates for Twitter will grant you the access to the following functionalities, grouped by types:
The templates are organized as described above. In each folder you’ll find the views provided for all this functionalities and two subfolders, one for the data sources and another one for the base views, which are more complex and will contain more info than the final views (and might be needed in some specific scenarios).
In the Configuration section you’ll find how to generate your own Denodo Templates for Twitter.
The Denodo Templates for Twitter require:
In order to get the parameters needed to authenticate to your Twitter account from Denodo, you need to follow these steps:
Now, you have all the data needed to configure the .properties file.
Before importing the VQL templates in Denodo, you have to set the following parameters defined in the denodo_templates_twitter.properties file:
twitter_templates_denodo.properties
Note: Make sure you leave an empty line at the end of the properties file.
Once you have completed all the previous steps of this configuration section, you finally can load the denodo_templates_twitter.vql file into Denodo. You only have to import it, along with the .properties file, using the Import option of the VDP Administration Tool:
After the import process finishes, refresh the Server Explorer by selecting File → Refresh and you will be able to see the data source and base views for accessing Denodo Templates for Twitter:
Denodo for Twitter views and folders structure
Now, you can execute queries over the Twitter views that have been created.
The standard API rate limits are described in the table in this link. Note that endpoints not listed in the chart default to 15 requests per allotted user. All request windows are 15 minutes in length. These rate limits apply to the standard API endpoints only, does not apply to premium APIs. Note that some of the views included in the Denodo Templates for Twitter may require an amount of requests that might exceed the limits of your Twitter account under some circumstances.
To be able to get more than the results limited by the Twitter API for each view, we can use pagination, a feature included in Denodo 7 but not available in Denodo 6.
This way, we can get the amount of desired data as long as we don’t reach Twitter API limits.
Although in Denodo 7 pagination can be done over JSON data sources, Twitter has set some limitations in their default API. For some of the endpoints pagination is not supported. You can see a more detailed explanation of the limitations in this link.
To configure pagination, you have to access the data source configuration and set up the following parameters in the Pagination section:
Here are the limitations in the Standard API according to this pagination context:
Resource |
Supports pagination |
Max “Count” value |
blocks/list |
Yes |
5000 |
mutes/users/list |
Yes |
5000 |
followers/list |
Yes |
200 |
friends/list |
yes |
200 |
lists/list |
No |
Unsupported |
lists/memberships |
Yes |
1000 |
lists/ownerships |
Yes |
1000 |
search/tweets |
No |
100 |
statuses/retweets_of_me |
No |
100 |
statuses/home_timeline |
No |
200 |
statuses/mentions_timeline |
No |
200 |
statuses/user_timeline |
No |
200 |
favorites/list |
No |
200 |
direct_messages/events/list |
Yes |
50 |
Note that all the resources that support pagination are configured to use it, so in case you use an enterprise/premium API you'll be able to paginate your queries.