• User Manuals »
  • Denodo Templates for Twitter - Quick Use Guide

Denodo Templates for Twitter - Quick Use Guide

Download original document


Overview

Twitter is a social networking site that relies on micro-blogging for communication. By typing short, 140 character messages (up to 280 since 2017) users can communicate with their followers and get information spread around. Since its inception, Twitter has grown and turned into a massive site used by mainstream media as well as average individuals to communicate about important issues, random issues, or just about what they’re doing at any time. It allows people to blog in short bursts, a practice also known as microblogging.

Twitter lets you make short posts about anything. These short posts are known as “tweets”, in reference to the chirping of birds, and appear chronologically on your Twitter profile page. In addition to posting, you can also follow others on Twitter to get their latest tweets in your Twitter feed. Conversely, if someone follows you, they get your tweets in their feeds.

Denodo Templates for Twitter

Importing the Denodo Templates for Twitter will grant you the access to the following functionalities, grouped by types:

  • Direct Messages
  • Direct Messages Events List
  • Lists
  • Lists List
  • Membership Lists
  • Ownership Lists
  • Tweets
  • Favorites List
  • Search Tweets
  • Home Timeline
  • Mentions Timeline
  • User Timeline
  • Retweets of Me
  • Users
  • Blocks List
  • Followers List
  • Friends List
  • Mutes Users List
  • Utils
  • Application Rate Limit Status

The templates are organized as described above. In each folder you’ll find the views provided for all this functionalities and two subfolders, one for the data sources and another one for the base views, which are more complex and will contain more info than the final views (and might be needed in some specific scenarios).

In the Configuration section you’ll find how to generate your own Denodo Templates for Twitter.

Requirements

The Denodo Templates for Twitter require:

  • When used with Denodo 6.0: At least the Denodo update 20181120.
  • When used with Denodo 7.0: At least the Denodo update 20190205.

Configuration

Get OAuth1 credentials

In order to get the parameters needed to authenticate to your Twitter account from Denodo, you need to follow these steps:

  1. You must have a Developer Account in order to authenticate to the Twitter API. Navigate through https://developer.twitter.com/en/apps/ and select the Application that you wish to configure with Denodo by clicking in the Details button.

  1. Now you have to go to the Keys and Tokens section. Those keys are what you need to get your templates working into Denodo with your Twitter App. Also, make sure that you have Read, write, and direct messages permissions, otherwise configure them in the Permissions tab.


 

Now, you have all the data needed to configure the .properties file.

Configure denodo_templates_twitter.properties file

Before importing the VQL templates in Denodo, you have to set the following parameters defined in the denodo_templates_twitter.properties file:

  • api_standard_version: the version you want to use of the Twitter API. By default, it’s set to v1.1.

  • client_id: Matches API key, obtained in the step 2 of the “Get OAuth1 credentials” section.

  • client_shared_secret: Matches API secret key, obtained in the step 2 of the “Get OAuth1 credentials” section.

  • access_token: Matches Access token, obtained in the step 2 of the “Get OAuth1 credentials” section.

  • access_token_secret: Matches Access token secret, obtained in the step 2 of the “Get OAuth1 credentials” section.

  • api_search_url: For the search functionality Twitter provides Standard, Enterprise and Premium APIs, so you can set this parameter using one of them. By default, this parameter is set to use Standard API.

twitter_templates_denodo.properties

Note: Make sure you leave an empty line at the end of the properties file.

Import the VQL file into Denodo

Once you have completed all the previous steps of this configuration section, you finally can load the denodo_templates_twitter.vql file into Denodo. You only have to import it, along with the .properties file, using the Import option of the VDP Administration Tool:

Refresh

After the import process finishes, refresh the Server Explorer by selecting File → Refresh and you will be able to see the data source and base views for accessing Denodo Templates for Twitter:

        Denodo for Twitter views and folders structure


Query

Now, you can execute queries over the Twitter views that have been created.

Limitations

Twitter API Limits

The standard API rate limits are described in the table in this link. Note that endpoints not listed in the chart default to 15 requests per allotted user. All request windows are 15 minutes in length. These rate limits apply to the standard API endpoints only, does not apply to premium APIs. Note that some of the views included in the Denodo Templates for Twitter may require an amount of requests that might exceed the limits of your Twitter account under some circumstances.

Denodo 6 doesn’t support pagination in JSON data sources

To be able to get more than the results limited by the Twitter API for each view, we can use pagination, a feature included in Denodo 7 but not available in Denodo 6.

This way, we can get the amount of desired data as long as we don’t reach Twitter API limits.

Pagination in Denodo 7 and later

Although in Denodo 7 pagination can be done over JSON data sources, Twitter has set some limitations in their default API. For some of the endpoints pagination is not supported. You can see a more detailed explanation of the limitations in this link.

To configure pagination, you have to access the data source configuration and set up the following parameters in the Pagination section:

  • Parameter in URL for page size: count
  • Page size: <number> (Represents the number of elements to return per page.)
  • Token continuation
  • Parameter in URL for “next” token: cursor
  • Path to “next” token in response: next_cursor

Here are the limitations in the Standard API according to this pagination context:

Resource

Supports pagination

Max “Count” value

blocks/list

Yes

5000

mutes/users/list

Yes

5000

followers/list

Yes

200

friends/list

yes

200

lists/list

No

Unsupported

lists/memberships

Yes

1000

lists/ownerships

Yes

1000

search/tweets

No

100

statuses/retweets_of_me

No

100

statuses/home_timeline

No

200

statuses/mentions_timeline  

No

200

statuses/user_timeline

No

200

favorites/list

No

200

direct_messages/events/list

Yes

50

Note that all the resources that support pagination are configured to use it, so in case you use an enterprise/premium API you'll be able to paginate your queries.