Appbase.io News

Database service for the #realtimeweb

Building meetupblast: An app for the most happening meetups

Meetup is an awesome app to find interesting events around you. Today, we take the core aspect of meetup - recommending events and make it social by showing a live feed of meetups that other people are attending.

Image: Screenshot of meetupblast, click on the image to see the live demo.

Or continue reading to see how we built this.

Key Ingredients

The meetupblast recipe can be broken into two key pieces: the backend worker and the user interface.

  1. Backend Worker:
    • We use the Meetup's long-polling RSVP stream endpoint to get realtime RSVPs.
      Note: This demo wouldn't have been possible without meetup supporting an API endpoint to extract the RSVP information.
    • We then store this data in appbase.io, which provides us a convenient way to query both historical data and realtime feeds - we call it the streaming DB ;).
  2. User Interface:
    • Querying appbase.io for meetup RSVP feeds by cities and topics.
    • The UI / UX logic maintained with a jQuery frontend. And we use typeahead for displaying the search interface.

What is meetupblast?

Meetupblast shows live RSVPs from people across the globe. It has some neat filters that allow you to select cities and the topics of interest, to see what meetups are happening in your city.

showing a filtered view

Image: Let's check for RSVPs in the all the singles tagged meetups in San Francisco.

Now when a person RSVPs for a singles meetup, you would see it show up on the meetupblast feed. Now don't get any ideas for stalking unsuspecting people. It's a cool way to find the most happening meetups!

A fun fact: While building this app, we noticed that there are about 120 RSVPs on meetup.com every minute, that's 2 every second. Or in 30 days, we would see over 5 million RSVPs - that's a good data set size to query in realtime ;).

Deep Dive

Now that we know what meetupblast does, let's get into the meat of how the app works.

Backend Worker

Our backend worker is a simple Node.JS code that keeps running forever on a digitalocean droplet.

Let's see - How does our worker consume meetup RSVP data and then store it into appbase.io?

http.get(meetup_url, function(res) {  
    res.on('data', function(chunk) { // 'data' event handler is called everytime there is a new meetup RSVP
        var data = JSON.parse(chunk);
        meetup_data.push(data);  // capture the RSVP responses in an array
    });
});
appbaseRef.index({  
    type: DATA_TABLE,            // collection to store the data into
    body: meetupData[dataCount]
}).on('data', function(res) {
    console.log("successfully indexed one meetup RSVP");
})

Meetup provides us a JSON object for every RSVP. We then write this data into appbase.io as soon as it arrives. appbase.io is built as a streaming layer on ElasticSearch, and provides a continuous query interface for streaming JSON results.

A RSVP JSON data looks like this:

"visibility": "public",
"response": "yes",
"guests": 0,
"member": {
    "member_id": 185034988,
    "photo": "http://photos1.meetupstatic.com/photos/member/d/0/0/0/thumb_248333248.jpeg",
    "member_name": "Wilma"
},
"rsvp_id": 1566804180,
"mtime": 1440266230993,
"event": {
      "event_name": "Wednesday Badminton @ Flying Dragon",
      "event_id": "224809211",
      "time": 1440630000000,
    "event_url": "http://www.meetup.com/Richmond-Hill-Friends/events/224809211/"
},
"group": {
    "group_topics": [
    {
        "urlkey": "social",
        "topic_name": "Social"
    },
    {
        "urlkey": "board-games",
        "topic_name": "Board Games"
    },
    {
        "urlkey": "movie-nights",
        "topic_name": "Movie Nights"
    }
    ],
"group_city": "Richmond Hill",
"group_country": "ca",
"group_id": 1676043,
"group_name": "Richmond Hill Friends",
"group_lon": -79.4,
"group_urlname": "Richmond-Hill-Friends",
"group_state": "ON",
"group_lat": 43.84
}

User Interface

The User Interface is a small frontend that queries appbase.io for realtime meetups based on specific cities and tags and displays them in a neat list.

Image: Showing meetups by city and topic tags - try it live.

The app directory structure looks like this:

meetupblast/  
|_ _ _ _ app/
|        |_ _ _ _ app.js
|        |_ _ _ _ request.js
|        |_ _ _ _ variable.js
|_ _ _ _ index.html

The codebase can be accessed at the meetupblast github repo.

app.js

This is the entry point of the codebase and brokers responsibility to variable.js and request.js.

request.js

Creates a meetup_request prototype which is responsible for three different queries.

  1. Fetching the list of the 1000 most popular cities and topic where meetups happen,
  2. A realtime query fetching live meetup data from the selected cities and topics,
  3. Pagination showing older RSVPs when a user scrolls to the bottom of the meetup feed.

variable.js

Instantiates a meetup prototype object which is responsible for the tag selection logic when a user ticks a city / topic and displaying a new meetup RSVP inside the feed.

A sneakpeak into continuous queries

Here's a sneakpeak into how the continuous query for fetching live meetup data works.

We use the appbase-js library (available via bower and npm) for making continuous queries over the meetup feed data.

appbaseRef = new Appbase({  
    url: "https://qz4ZD8xq1:a0edfc7f-5611-46f6-8fe1-d4db234631f3@scalr.api.appbase.io",
    appname: "meetup2"
})
appbaseRef.streamSearch({  
    type: "meetup",
    size: 100,
    body: {
      "query": {
        "filtered": {
          "filter" : {
            "terms" : { 
              "group_city_simple" : ["San Francisco", "Ahmedabad"]
            }
          }
        }
      }
    }
}).on('data', function(res) {
   if (res.hits.hits) {
      // feeds are returned in the field 'res.hits.hits'
      console.log("initial feed size: ", res.hits.hits.length);
   } else {
      // realtime updates with one new RSVP
      console.log("new data feed: ", res._source);
   }
})

streamSearch() provides a data handler that returns:
1. the matching records to our query of find meetups with Social or Singles topics,
2. New records as RSVPs in Social or Singles topics arrive.

Summary

We created meetupblast as a way to find happening meetups by looking up what other people are RSVPing.

  1. We use the Meetup's long-polling API to get the live data of meetups.
  2. We use appbase.io to transform this into queriable live data.
  3. We put together a small frontend that allows neat filtering based on cities and topics.

If we wished, we could have additionally filtered by date range, geo queries - basically any field supported by the Meetup's RSVP data format.

Without further ado, here are the important links: meetup worker code, meetupblast and the demo.

Author image
Founder at Appbase, read my musings on the db world.