Posted on by Jon

Herding Code 125: Truffler with Joel Abrahamsson, Marcus Granstrom and Henrik Lindstrom

On this episode of Herding Code, the guys talk to Joel Abrahamsson, Marcus Granström and Henrik Lindström about Truffler, a solution for building advanced search and querying functionality for websites and other data-centric systems.

  • They talk about their backgrounds and combining their different skills to build something pretty awesome.
  • K. Scott says Truffler has a REST API and can be called from several languages (.NET, Java, js). How to send queries, where is the data?
  • Joel explains they’re using Elastic search, which indexes JSON documents; you push data to it in the form of JSON, then query it using JSON. The data is stored, but the point is to build awesome search.
  • Marcus points out you can get a Truffler in a box, where you get a server to your house so you can run it inside your own internet if you like.
  • K. Scott mentions the examples using the C# API on the Truffler home page. Do you convert that to a URL for your service?
  • Joel talks about using Elastic Search and the bits they’ve added on top of it, including the various integrations or client APIs they are making available.
  • K. Scott says he ran through their example; was able to just bang his way through it without having to dig into the documentation too much.
  • Joel: Instead of bringing your data to the search engine, we’re trying to bring the search engine to you, so you can query it in a way that feels natural in C#. That’s the whole point of Truffler.
  • Jon Asks about being able to search for a keyword and also have a geographical search and how that works.
  • Joel says they search for the keyword and then filter by coordinates, which does not affect relevance. Can specify that matches for the keyword have double the relevance of non-matches.
  • Talking about the ability to modify relevance and rank criteria higher and lower.
  • K. Scott asks if they contribute any patches or make any patches or augment anything to the technology they are building on?
  • Henrik has gone through the elastic search source code quite a few times. Found bugs, but not trying to augment it, just trying to find a way to package it in a nice way.
  • Jon asks about the document search capabilities (pdf, xml, word, etc) – part of elastic search? Or did you have to do some of that work?
  • Marcus replied that most features are basically from the elastic search core functions. The mods we have are just bug fixes. Not exposing any new functionality, just providing some features that are a little different from the core project.
  • They’re trying to make search easier so you don’t have to analyze and then index your data; they’ll do that for you.
  • Jon asks about highlighted words and bacon. Joel says it’s configurable. Can ask for one big fragment with highlighted keywords, or several small fragments, comes back as a separate fields in the JSON document. To make it as easy as possible, they are using LINQ syntax. Example: Select.As Highlighted()
  • Discussing how to add search to your website/blog.
  • They’re using JSON.NET. Discusses client class and using extension methods.
  • They discuss how to do a Google-type search – typing in textbox, brings results.
  • K. Scott: Any support for Word and PDF? Joel: Yes, it’s out of the box with Elastic Search; handles all kinds of formats.
  • K. Scott asks what the biggest challenge is. Joel: Everything. Branding, building the product, figuring out the market.
  • K. Scott says this is actually a product and a business now; was that new for the three of you? Joel says yes; were forced into it when working for a customer together who needed to search and query data not stored in a normalized database. Was the initial seed idea of Truffler.
  • K. Scott asks about node. Henrik said they use node as a reverse proxy to do authentication for Elastic Search.
  • They know what it can do; they can control how it’s used. It’s scary and exciting to see how people are using it.
  • Marcus said have to know what queries they want before they come; they are adapting it to handle so many queries without knowing what people are going to use it for.
  • Jon asks about performance / caching results on client side. Joel says servers can handle a lot, but there’s always latency. With .NET, allow you to cache search queries, which are serialized. For Get requests, don’t have caching; they’re pretty quick, but will add them in the future.
  • K. Scott asks what’s next. Lots of features; very interested in feedback. Have partners/customers testing with large sets of data. Lots of things in the .NET API they don’t expose but would like to. Want to provide as much flexibility as possible.
  • K. Scott asks about storing his own metrics. Providing some metrics about how his search is being used? They’re working on that, and SSL, encrypted indexes in planning stage.
  • Jon asks about support for custom synonyms; they explain what that means.
  • Plans and prices – free for developer. Then have basic and premium plans depending on how much you’re using it, what features you want, etc.
  • Joel says they have support for inheritance.
  • Jon asks about pricing. With developer license, if have open source project, get quite a bit of functionality and features.
  • Joel says that for developers, they want to encourage use. Would like feedback.
  • Jon asks how this compares to other search engine options and other document databases like RavenDB? Joel says you can use it for a document database, but that’s not the primary purpose of it. Raven is an awesome document database with text search, where this is awesome text search with basic document database capabilities. Henrik says if you take hardcore search, their main idea is not to provide hardcore search, but to enable developers to utilize search without being search experts.
  • Released client for Episerver, used widely in Sweden. Truffler will take care of indexing and hook up to events for you. They have the concept of filtering (using the LINQ where method), and have the ability to extend that.
  • K. Scott asks if this is open source. Joel says not exposing the source code at the moment. The .NET API is a very important part of their product, do some really cool things with it, would like to keep it to themselves. May release it in the future.
  • Truffler web site is truffler.net. On the about page, there are links to twitter and Joel’s blog.

 

Show Links:

Download / Listen:

Herding Code 125: Truffler with Joel Abrahamsson, Marcus Granstrom and Henrik Lindstrom

[audio://herdingcode.com/wp-content/uploads/HerdingCode-0125-Truffler.mp3]

This week’s show notes were typed up by @RobinDotNet – Thanks!!!

CategoriesUncategorized