Welcome to the land of SCALR, where everyone smiles and the sun shines bright.
Our beloved Hobo Lobo is chalking a 'SCALR' 101.
Today, we are excited to announce the SCALR API in preview.
Key highlights of the API:
1. SCALR = scale: SCALR is designed from ground to scale your production grade apps.
2. Data Streams 2.0: Appbase.io now supports streams on database queries and filters with the same simplicity of streams on messaging.
3. JSON schema: We've listened to all the user feedback and SCALR API is based on a JSON schema, unlike the previous APIs which were based on a graph datamodel.
4. API Compatible with ElasticSearch: Appbase works out of the box with all ElasticSearch APIs, client libraries, and toolings.
As the name suggests, our prime focus with this iteration of the API has been achieving a massive scale of API operations, to the tune of 100,000 writes and reads per second.
You might wonder - why is addressing scale so important? A typical successful application would never exceed a throughput of a few hundred requests per second and generally be okay with serving responses in time.
So much complexity in software comes from trying to make one thing do two things.
This quote from Ryan Singer especially sings true when one thinks of scaling database operations. So much of the software service complexities stem from keeping up with the infrastructure needs, and end up becoming burning pain points. It should come as no surprise that most mainstream distributed systems were born out of the scaling pain points faced by the 0.01% of the applications.
Not only does addressing scale solve the burning pain points faced by the 0.01%, it provides confidence around the edge-cases and leads to faster adoption in production environments and success stories. We hope all of these come true for SCALR. In building SCALR, focusing on scale has lead us to a better foundational design, a more rigorously tested codebase, and a fine-grained monitoring of our deployment infrastructure. We'll go more in depth about these learnings over the next few months, but most importantly - for the 99.9% folks reading this,
SCALR has tons of improvements and features over
git diff changes --human --pretty-print
SCALR - What's in a name
The most tangible change is in the codename -
scalr. Breaking the convention of keeping a dull version name like
v2.0, we will go with the version
scalr. Docs for scalr are available at http://docs.appbase.io/#/scalr/ and the API base URL would be
2. Data Streams 2.0
Data Streams aka realtime events are rethought in
SCALR. Instead of being a websockets based extension of the API to track just the document and reference changes, data streams are deeply baked into the
SCALR API. Data Streams are implemented over http-streaming and are pervasive over web, mobile and language native libraries. Data Streams can stream results of queries and filters as new data comes into the system.
3. Adieu to Graph datamodel
SCALR the JSON based schema-free datamodel of ElasticSearch. This is a major change over the graph datamodel extensively used by the
v2 API. The move is aimed at simplicity and to provide a less opinionated data model. True graph relationships can still be modeled in
4. Compatibility with ElasticSearch
scalr is compatible out of the box with the ElasticSearch APIs. This means our users can take advantage of the vast plethora of client libraries made available by the folks at Elastic, have access to a vibrant open-source community and have the freedom to import / export data from Appbase.io conveniently.
4.a. HTTP Basic Authentication
scalr uses HTTP Basic Authentication for securing app access (similar to Github and Wordpress's authentication mechanisms). Every Appbase app can have one or more access tokens (username, password) with granular read and write permissions to allow fine-grained access. ElasticSearch's access tokens work in the same way, a major compatibility win.
Note: This is a change from the earlie r use of an authentication header
4.b. What's not compatible?
The APIs related to devops, like clusters, multi-index operations, index creation are not a part of Appbase.io.
How does this affect our roadmap going forward?
SCALR API's design is robust. We will be making a lot of backwards-compatible changes to augment the API and our docs.
- We will be launching the SCALR API as the official version over the next month. In the meanwhile, we will be helping with code migration and data modeling guides for users of the
- Data Streams 3.0: A unified streams API that includes support for realtime aggregations. Imagine being able to build somthing like the live Twitter Trends with a single API ;-).