Serverless search on AWS Lambda with Janet and SQLite

11 Jan 2022

Recently I have been porting a simple app on AWS Lambda for posts and comments to use the Janet programming language. A previous post describes how Janet is a great fit to use with AWS Lambda. The web app now also supports search through the search bar in the header. An earlier version of the postings app was written in Clojure and had a search functionality backed by ElasticSearch/OpenSearch on AWS, which is described in this post.

Since postings is a demo app that is rarely used, it benefits from the serverless quality of being able to scale the compute resource usage down to zero. The storage resources are also minimal. Sadly OpenSearch does not follow this serverless model by requiring a provisioned cluster.

The search is now backed by an SQLite database with full text search (fts5) database on an Elastic File System (EFS). A good source to get started with EFS and AWS Lambda is here: Unlocking more serverless use cases with EFS and Lambda on the Lumigo blog.

The postings app now consists of two AWS Lambda functions. One to handle all the web requests and one Lambda function in a VPC with the SQLite database, which imports all the data from the main DynamoDB database and handles all search requests.

By using SQLite on EFS for the search, the posting app became even more serverless-y by being able to scale down to zero compute and almost zero storage resource usage.

Postings web app at https://postings.thegeez.net, source at https://github.com/thegeez/janet-postings