The gang talks to Keith Horwood about stdlib, nodal, and lots more!
Download / Listen: Herding Code 223: Keith Horwood on StdLib, Nodal, and Functions as a Service
Show Notes:
- Standard Lib (stdlib)
- (00:44) Standard Lib is a registry for serverless microservices. It’s kind of likeĀ a mix between npm and heroku, so there is a central registry, but rather than just installing the services locally, they handle deployment for you. There are command line tools available via npm that make it easy to create (lib create)Ā and deploy microservices (lib up).
- (02:23) Jon says he’s pretty impressed with the interactive experience on the website, where you can very quickly deploy a service. Keith explains how they set up a service that handles the tarball packaging to allow creating a service in the browser. They expect that developers will usually start with the cli tools, but the browser based onboarding experience is nice for new users to get familiar with the service.
- (04:22) K Scott says that essentially he can just write a javascript function and have it running as a service. Keith agrees, but says it’s also set up so you don’t need to manage servers – it automatically scales and self-heals.
- (05:34) K Scott asks about how the monitoring and exception handling works. Keith says they work with several monitoring systems. Currently the output is just a text dump, but they’re working to improve that.
- (06:15) K Scott asks about how you handle persistance. Keith says that currently they don’t offer their own persistence layer, and he recommends just using compose or Dynamo or RDS. They’re talking to some companies doing some neat stuff with GraphQL and gives a shoutout to Graphcool who runs a really cool GraphQL backend as a service.
- (07:48) K Scott asks about their pricing model. Keith says they’re going to be going with a Twilio-style system where you fill up a wallet and pay per compute, so you only pay for what you use.
- (09:09) K Scott asks how they differentiate from AWS Lambda and Azure Functions. Keith explains how they’re providing another layer of abstraction. The existingĀ infrastructure and infinite scalability are great, but the workflows are too complex, so stdlib provides a nice layer on top of that.
- (10:42) K Scott asks about their versioning system, in which development is mutable but releases are immutable. Keith explains that service immutability is a point of trust for API consumers. When you’re developing, you’re not supporting consumers, so you can continue to change your service, but once you deploy your service can’t be changed.
- (12:42) K Scott asks how clients will reference anĀ API byĀ version. Keith talks about the syntax for referencing a specific version and says if you don’t specify a version number you’ll just get the latest.
- (13:55) Jon asks what to do if he deploys a package with a really bad error. Keith explains how you can use “lib down” on a package, and discusses the monitoring and notification systems they’ve got in place to communicate with consumers.
- (15:17) Jon asks about the view templates. Keith says that they started building view templates for internal use as backends to single page applications, then realized they’d be very useful to other developers. They also have templates for Alexa apps including a Delores Abernathy (WestWorld) sample app.
- (16:44) K Scott asks about what some of the biggest challenges they faced putting it together. Keith says that AWS Lambda can be a black box and talksĀ about the process ofĀ finding the right developer abstraction.
- (18:36) K Scott asks about Keith’s blog postĀ titledĀ Using “Server-less” Architecture to Massively Parallelize DNA Sequence Alignment via StdLib and Node.js. Keith explains what DNA sequence alignment is, and how you can use massive parallelization to go from an n-squared problem to operate with a time complexity of one.
- NtSeq
- (21:35) Jon asks about Keith’s NtSeq library for DNA sequencing. Keith actually dropped out of grad school in bio chem, so he had the background and some code lying around.
- Collaboration, Dependencies, and Documentation
- (22:50) Kevin says that a lot of function as a service samples are petty simple, but real solutions will require multiple collaborative services. Are there thoughts on how to assemble collaborating services effectively? Keith explains that these were some of the design goals for stdlib.
- (25:22) Kevin asks if there are ways to track service dependencies. Keith talks about the static analysis opportunities, and mentions thatĀ all dependent services you’ve been calling will show up in your dashboard.
- (26:15) JonĀ is pretty impressed with the service documentation and asks how it’s created. Keith talks about the markdown and service json based documentation.
- nodal
- (27:25) K Scott asks about what prompted Keith to create stdlib. Keith talks about the history, starting with the nodal platform. nodal has been pretty popular as a platform, but many really liked the workflows for deployment and service management, but didn’t necessarily want to learn a new platform, and many didn’t have data persistence needs.
- (29:40) K Scott says that he finds the Nodal sample very easy to read – ES2015 classes that extend a base controller. Keith talks about the unique time in JavaScript history when Nodal started. He’d made a bet on io.js during the Node / io.js split, and had leveraged ES2015 syntax. When io.js and Node merged back up, Nodal was the one of the first mature Rails-like frameworks.
- (31:35) K Scott asks about the ORM. Keith talks about why he built the query composer. Nodal’s ORM is called the composer, and it uses method chaining, then reduces everything down to one query. That gets around the n+1 issues you’ll run into with ORMs like Active Record. There’s a GraphQL example at graphql.nodal.com which can take a GraphQL query and translate it down to a single Postgres query. They’re looking at breaking the composer out to allow use in other GraphQL applications. K Scott asks more about the join syntax and lazy / eager loading.
- Ā (35:21) K Scott asks about the extends keyword and object orientation. Keith says that the JavaScript community is growing so quickly that as long as developers can read the code easily, they’re just happy that it’s there. A lot of these newer developers aren’t that opinionated, and developers are happy to work with opinionated frameworks.
- What do you do for fun, music apps, what’s next?
- (36:35) K Scott asks Keith what he does when he’s not working on stdlib and nodal. Keith says it’s the majority of his time now. Keith says he used to run.
- (37:28) Jon noticed Keith’s audiosynth.js lib on GitHub. Keith says that audiosynth was done before webaudio and webmidi, and he stumbled on someone who was generating simple sine wave audio files in JavaScript, and he extended it to use some more interesting string synthesis algorithms to make some more advanced tones. There are some more advanced audio APIs available now. Keith says that the most interesting outcome of that work was the connection he made to the first engineer they hired at stdlib.
- (40:22) K Scott asks what’s coming up. Keith talks about the authentication and authorization layers as well as multi-language SDKs and support that are coming out soon. K Scott asks about the current auth story, and Keith says that currently you code it all yourself.