Appbase.io News

Database service for the #realtimeweb

SCALR API is in preview now

Welcome to the land of SCALR, where everyone smiles and the sun shines bright.

Our beloved Hobo Lobo is chalking a 'SCALR' 101.

Today, we are excited to announce the SCALR API in preview.

Key highlights of the API:
1. SCALR = scale: SCALR is designed from ground to scale your production grade apps.
2. Data Streams 2.0: Appbase.io now supports streams on database queries and filters with the same simplicity of streams on messaging.
3. JSON schema: We've listened to all the user feedback and SCALR API is based on a JSON schema, unlike the previous APIs which were based on a graph datamodel.
4. API Compatible with ElasticSearch: Appbase works out of the box with all ElasticSearch APIs, client libraries, and toolings.

SCALR equals Scale

As the name suggests, our prime focus with this iteration of the API has been achieving a massive scale of API operations, to the tune of 100,000 writes and reads per second.

You might wonder - why is addressing scale so important? A typical successful application would never exceed a throughput of a few hundred requests per second and generally be okay with serving responses in time.

So much complexity in software comes from trying to make one thing do two things.
-Ryan Singer

This quote from Ryan Singer especially sings true when one thinks of scaling database operations. So much of the software service complexities stem from keeping up with the infrastructure needs, and end up becoming burning pain points. It should come as no surprise that most mainstream distributed systems were born out of the scaling pain points faced by the 0.01% of the applications.

Not only does addressing scale solve the burning pain points faced by the 0.01%, it provides confidence around the edge-cases and leads to faster adoption in production environments and success stories. We hope all of these come true for SCALR. In building SCALR, focusing on scale has lead us to a better foundational design, a more rigorously tested codebase, and a fine-grained monitoring of our deployment infrastructure. We'll go more in depth about these learnings over the next few months, but most importantly - for the 99.9% folks reading this, SCALR has tons of improvements and features over v2.


The Details

git diff changes --human --pretty-print

1. SCALR - What's in a name

The most tangible change is in the codename - scalr. Breaking the convention of keeping a dull version name like v2.0, we will go with the version scalr. Docs for scalr are available at http://docs.appbase.io/#/scalr/ and the API base URL would be https://scalr.api.appbase.io.

2. Data Streams 2.0

Data Streams aka realtime events are rethought in SCALR. Instead of being a websockets based extension of the API to track just the document and reference changes, data streams are deeply baked into the SCALR API. Data Streams are implemented over http-streaming and are pervasive over web, mobile and language native libraries. Data Streams can stream results of queries and filters as new data comes into the system.

3. Adieu to Graph datamodel

SCALR the JSON based schema-free datamodel of ElasticSearch. This is a major change over the graph datamodel extensively used by the v2 API. The move is aimed at simplicity and to provide a less opinionated data model. True graph relationships can still be modeled in SCALR.

4. Compatibility with ElasticSearch

scalr is compatible out of the box with the ElasticSearch APIs. This means our users can take advantage of the vast plethora of client libraries made available by the folks at Elastic, have access to a vibrant open-source community and have the freedom to import / export data from Appbase.io conveniently.

4.a. HTTP Basic Authentication

scalr uses HTTP Basic Authentication for securing app access (similar to Github and Wordpress's authentication mechanisms). Every Appbase app can have one or more access tokens (username, password) with granular read and write permissions to allow fine-grained access. ElasticSearch's access tokens work in the same way, a major compatibility win.

Note: This is a change from the earlie r use of an authentication header Appbase-Secret.

4.b. What's not compatible?

The APIs related to devops, like clusters, multi-index operations, index creation are not a part of Appbase.io.

Roadmap

How does this affect our roadmap going forward?

SCALR API's design is robust. We will be making a lot of backwards-compatible changes to augment the API and our docs.

  1. We will be launching the SCALR API as the official version over the next month. In the meanwhile, we will be helping with code migration and data modeling guides for users of the v2 and v3 APIs,
  2. SCALR already comes with client libraries for Node.JS, Javascript and Go. Over the next months, we will be building client libraries for Java, Ruby, Python, Android, and Swift.
  3. Data Streams 3.0: A unified streams API that includes support for realtime aggregations. Imagine being able to build somthing like the live Twitter Trends with a single API ;-).
Author image
Founder at Appbase, read my musings on the db world.