Posts by /u/everythingserverless

I’m building 100% Serverless Elasticsearch

I’m an SDE at AWS and for a while now I’ve been working in my spare time on a side project to build a fully Serverless search engine. I decided to target Elasticsearch, being the most used search engine database out there. I am determined to fix, or at least improve where I can, on the following issues:

  1. Elasticsearch is incredibly annoying to configure. There are hundreds of settings and configurations to choose, and many of them are one way doors – for example, once you configure your shard count you cannot change it without a full migration to reindex your data into a new index.
  2. Similar to anything that isnt serverless, having to constantly monitor and worry about CPU, memory pressure and other metrics is something no one has time for in the real world. Elasticsearch has launched an autoscaling policy feature, but this will still hit issues for users who have big jumps in usage and leaves yet another complexity for the user to configure.
  3. Again, common to serverless is the issue of developers having to over-provision their instances and have unused capacity just sitting around – seems like a loss for everyone involved.

I thought I’d reach out to the community to see if anyone like me would like to see Serverless Search and whether they would find it useful in their applications.

submitted by /u/everythingserverless
[link] [comments]

Read More