Groovy coding

Setup

· Copy the project CO7X17.miniproject1 under partI in the local copy of your private repository.

· Import the project into your workspace using Gradle Buildship 2.

Migration process setup

There are several clients for the Twitter API that allow us to query data on Twitter. We are going to make calls to the REST API directly via twurl copying data in bulk once.

The data migration process is achieved by executing from a shell terminal the command ./twitterMigrate.sh. The script will show a URL which should be pasted on to a browser and the PIN obtained should be pasted on the terminal. Once authorization is successful, friends and follower's tweets will be filtered and copied onto your MongoDB database on mLab (once you finish exercise 1).

The script will not work straight away. Please configure it as follows:

· Installation process:

· A Bash shell is needed for executing the script ./twitterMigrate.sh. If you are working on Windows, follow  these instructions .

· Install twurl, to connect to the Twitter REST API, as described  here . You may need to install  Ruby Gems .

· Install jq, to pretty print JSON in files (optional), as described  here

· if you don't want to install it, remove | jq . from the commands in the script ./twitterMigrate.sh

· Configuration:

· Configure the data store connection as done for the lab2 exercise in class miniproject1.mongoDb.MongoDB

· Do exercise 1.

· Execution: the script can be executed by using the command ./twitterMigrate.sh on a Bash shell.

· By default it uses an upper bound of 10 records when calling the Twitter API (i.e. return the first 10 friends or the first 10 tweets).

· This can be changed by passing a number as argument to the script, e.g. ./twitterMigrate.sh 20.

Exercise 1: fetching data from Twitter [30 marks]

In class miniproject1.mongoDb.BulkTransfer, implement the code necessary to transfer data from the stored JSON files in your MongoDB instance on mLab. The database should contain two collections of documents:

· friends: containing information about friends and their tweets.

· followers: containing information about followers and their tweets.

Take into account the following requirements:

· Denormalization is going to be applied and a user may be stored in both collections and, therefore, their tweets will also be stored twice.

· Each user (friend or follower) must contain its tweets in a new field tweet.

· Store the following information about friends and followers

· id_str,

· name,

· description,

· favorites_count,

· followers_count,

· friends_count,

· language,

· location,

· screen_name,

· url,

· utc_offset

· Store the following information about tweets

· id_str,

· created_at,

· entities,

· favorite_count,

· favorited,

· user.id_str,

· in_reply_to_screen_name,

· in_reply_to_status_id,

· in_reply_to_user_id,

· language_code,

· retweet_count,

· retweeted,

· source,

· text

To fetch JSON documents from Twitter for testing your scripts without pushing them to your MongoDB instance, comment the line ./gradlew migrateToMongoDb in the shell script twitterMigrate.sh using #, that is:

# ./gradlew migrateToMongoDb

Marking guidelines:

The migration process must be executable by running ./twitterMigrate 10 from a shell terminal. If there are compilation/runtime errors or this command cannot be executed for other technical reasons, a mark of 0 (zero) will be awarded.

The implementation of functional requirements will be marked as follows:

· Information about friends is migrated to mLab. [10 marks]

· Information about followers is migrated to mLab. [10 marks]

· The command ./twitterMigrate 10 executes without problems, doing what it is intended to do. [10 marks]

Exercise 2: followers report [10 marks]

In the controller IndexController, add code to the handler method followers in order to list all your follower that have tweets that mention martinfowler, i.e. screen_name=='martinfowler' for an entity user_mentions in a tweet published by your follower.

The controller must get the information from the data on your MongoDB instance on mLab, and not from the JSON files directly.

For the sample data provided in src/main/resources/twitter/, the result should be:

Marking guidelines:

The web app must be executable by using ./gradlew bootRun. If there are compilation errors or this command cannot be executed for other technical reasons, a mark of 0 (zero) will be awarded.

The implementation of functional requirements will be marked as follows:

· Followers are listed. [2.5 marks]

· Only those followers who mentioned 'martinfowler' in their tweets are listed. [7.5 marks]

Exercise 3: friends report [20 marks]

In the controller IndexController, add code to the handler method friends in order to mine some intelligence about your Twitter account:

For each friend, add an object FriendDto into the list friends with the following information:

· name: your friend name

· description: your friend description

· noTweets: their number of tweets (up to 10)

· noRetweets: the overall number of retweets (aggregated retweet count for all tweets) for their selected tweets (those fetched from twurl or in the sample files)

· noActiveTweets: the number of tweets that have caused discussion (whose whose retweet count is greater than 100) for their selected tweets (those fetched from twurl or in the sample files)

The controller must get the information from the data on your MongoDB instance on mLab, and not from the JSON files directly.

For the sample data provided in src/main/resources/twitter/, the result should be:

Marking guidelines:

The web app must be executable by using ./gradlew bootRun. If there are compilation errors or this command cannot be executed for other technical reasons, a mark of 0 (zero) will be awarded.

The implementation of functional requirements will be marked as follows:

· fields name, description and noTweets extracted correctly for up to 10 friends. [5 marks]

· field noRetweets extracted correctly for up to 10 friends. [7.5 marks]

· field noActiveTweets extracted correctly for up to 10 friends. [7.5 marks]