(01:20) Jeremy’s shops really like using document databases because there’s a lot less friction to develop and ship your applications. They were having trouble with their document databases in production, though, so when they saw Postgres has JSON support they decided to build a client library on top of it so they’d have the developer productivity of a document database, but with the stability and tools that come with Postgres.
(03:24) K Scott mentions Jeremy’s recent blog post comparing the simplicity of building applications using a document database as opposed to using an ORM. Jeremy explains how document databases allow you to have a more evolutionary design that’s able to handle changes.
(04:35) K Scott asks about their recent use of Marten as an event store. Jeremy says they’re building up view projections that express the read side state of the application. The advantage of having them in Marten is that they’re just documents, and you don’t need additional infrastructure on top of your application document database.
(06:07) K Scott remarks on the great documentation and API design. Jeremy says that after years of abuse over his StructureMap documentation, he’s decided to make documentation a much bigger emphasis for this project.
(06:45) K Scott asks about how the documents are stored in the database. Jeremy says each document type has an underlying table with some metadata columns, and the main document is in a single JSONB column. JSONB is a Postgres data type that’s a binary representation of a JSON document that’s much more efficient for searching and querying.
(08:12) Jon asks about differences between BSON, JSONB, etc.
(12:22) Jon asks if he’d looked at Entity Framework. Jeremy says that due to differences in data structures, it’s a pretty different model. There’s nothing that prevents you from using Marten for documents in a Postgres database alongside EF or Dapper for relational tables in the same database.
(13:51) K Scott asked where Marten falls in transactional support, since document databases take pretty different views on that. Jeremy says that Marten is ACID all the way down – you can query for a document as soon as it’s committed, and you don’t need to deal with eventual consistency unless you opt into it for event store.
(15:12) K Scott asks about the testing story. Jeremy talks about his shop’s integration testing strategy, and that document databases are a lot easier to integration test than relational databases. From a unit testing perspective, they do some minimal mocking, but he recommends integration testing rather than unit testing in most places.
(17:42) Kevin asks about JSON patching support, since Postgres has support for that.
(18:48) K Scott asks about their use of Sqitch. Jeremy explains how it’s a simple database migration system using SQL semantics.
(22:02) Jon asks about the release notes mention of lowering memory usage. Jeremy says the main work was in minimizing serializing to JSON strings and using object pooling.
(24:04) K Scott mentions the successful release and active Gitter channel. Jeremy says this is the smoothest experience he’s had with an open source release.
(25:05) Kevin asks if there are other libraries out there that are making it easier to work with Postgres, and asks if there’s any hope of standardization across Postgres, Oracle, and SQL Server.
(27:22) Scott K asks about the main selling points of Marten on top of Postgres.
(28:22) Scott K asks if they’ve looked at cloud document databases as well, like DocumentDB. Jeremy explains that it’s pretty tied to Postgres.
(29:40) Scott K asks about best practices on querying. Jeremy says they invested a lot of time in performance optimizations, and talks about calculated indexes, calculated fields for complex structures, and Marten’s compiled queries in C# code (which bypass compiling LINQ queries). The npgsql library they run on top of also supports batching, which can give you big boosts in chatty applications. There’s also support for including related fields in one database roundtrip.
(34:04) K Scott asks about Alba. Jeremy talks about the need their shop has for unit tests that run the full HTTP request cycle without hitting a webserver. This allows you to test the full end to end output for cases where you have filters, caching, middleware, etc. It was originally inspired by the Play framework in Scala.
(38:33) Jon asks when it makes sense to move from the ASP.NET Core test host to use Alba. Jeremy says that it’s mostly a matter of preference, but Alba does a good amount of legwork for you for things like routing.
(39:52) K Scott asks how Alba fits in with integration tests with Selenium. Jeremy says he’d really just want to use Selenium when there’s some significant UI functionality, but for the most part he’d avoid it as much as possible and use Alba.
(41:00) Jon asks about Jeremy’s experiences in porting to .NET Core. Jeremy says that the newer projects were straightforward, but his older projects that had a good amount of reflection were really hard. He’s sorry to see project.json go, and he things the dotnet CLI is worth upgrading just about on its own since it makes automation almost trivial.
Changes in the .NET Open Source world
(43:47) K Scott circles back to ask about Jeremy’s comments about Marten being the most positive experience he’s had with .NET open source. Jeremy says the .NET open source ecosystem is a lot more positive and helpful. Marten fills a pretty good niche that many people found helpful. Jeremy says that he’s seen the user community become a lot more collaborative compared to previously treating open source maintainers as product support. Jeremy says he also gets pull requests for documentation, which is something he’d never expect a few years ago.
(46:26) Kevin asks about Jeremy’s thoughts on turning StructureMap over to someone else. Jeremy says it’s been 14 years, people use it in ways he wouldn’t expect, ASP.NET Core has a built-in IoC Container, and it’s hard to make StructureMap work with the ASP.NET Core IoC requirements. The project is still viable and continuing, it’s just going to be better for everyone under new ownership.
(49:15) Frank asks on Twitter “Why so much hate for Scrum?” Jeremy talks about the ceremony and tracking in Scrum take all the joy out of Agile programming. Jeremy says Scrum is the Scrappy Doo of the Agile Programming world.
(54:05) K Scott asks about Jeremy’s post: Thoughts on Agile Database Development. Jeremy says that relational databases have long been the final frontier of agile development, and the database community has resisted agile development.
(56:42) K Scott asks what Jeremy’s up to, and Jeremy talks about his next release in May: a human.
(00:44) Standard Lib is a registry for serverless microservices. It’s kind of like a mix between npm and heroku, so there is a central registry, but rather than just installing the services locally, they handle deployment for you. There are command line tools available via npm that make it easy to create (lib create) and deploy microservices (lib up).
(02:23) Jon says he’s pretty impressed with the interactive experience on the website, where you can very quickly deploy a service. Keith explains how they set up a service that handles the tarball packaging to allow creating a service in the browser. They expect that developers will usually start with the cli tools, but the browser based onboarding experience is nice for new users to get familiar with the service.
(05:34) K Scott asks about how the monitoring and exception handling works. Keith says they work with several monitoring systems. Currently the output is just a text dump, but they’re working to improve that.
(06:15) K Scott asks about how you handle persistance. Keith says that currently they don’t offer their own persistence layer, and he recommends just using compose or Dynamo or RDS. They’re talking to some companies doing some neat stuff with GraphQL and gives a shoutout to Graphcool who runs a really cool GraphQL backend as a service.
(07:48) K Scott asks about their pricing model. Keith says they’re going to be going with a Twilio-style system where you fill up a wallet and pay per compute, so you only pay for what you use.
(09:09) K Scott asks how they differentiate from AWS Lambda and Azure Functions. Keith explains how they’re providing another layer of abstraction. The existing infrastructure and infinite scalability are great, but the workflows are too complex, so stdlib provides a nice layer on top of that.
(10:42) K Scott asks about their versioning system, in which development is mutable but releases are immutable. Keith explains that service immutability is a point of trust for API consumers. When you’re developing, you’re not supporting consumers, so you can continue to change your service, but once you deploy your service can’t be changed.
(12:42) K Scott asks how clients will reference an API by version. Keith talks about the syntax for referencing a specific version and says if you don’t specify a version number you’ll just get the latest.
(13:55) Jon asks what to do if he deploys a package with a really bad error. Keith explains how you can use “lib down” on a package, and discusses the monitoring and notification systems they’ve got in place to communicate with consumers.
(15:17) Jon asks about the view templates. Keith says that they started building view templates for internal use as backends to single page applications, then realized they’d be very useful to other developers. They also have templates for Alexa apps including a Delores Abernathy (WestWorld) sample app.
(16:44) K Scott asks about what some of the biggest challenges they faced putting it together. Keith says that AWS Lambda can be a black box and talks about the process of finding the right developer abstraction.
(21:35) Jon asks about Keith’s NtSeq library for DNA sequencing. Keith actually dropped out of grad school in bio chem, so he had the background and some code lying around.
Collaboration, Dependencies, and Documentation
(22:50) Kevin says that a lot of function as a service samples are petty simple, but real solutions will require multiple collaborative services. Are there thoughts on how to assemble collaborating services effectively? Keith explains that these were some of the design goals for stdlib.
(25:22) Kevin asks if there are ways to track service dependencies. Keith talks about the static analysis opportunities, and mentions that all dependent services you’ve been calling will show up in your dashboard.
(26:15) Jon is pretty impressed with the service documentation and asks how it’s created. Keith talks about the markdown and service json based documentation.
(27:25) K Scott asks about what prompted Keith to create stdlib. Keith talks about the history, starting with the nodal platform. nodal has been pretty popular as a platform, but many really liked the workflows for deployment and service management, but didn’t necessarily want to learn a new platform, and many didn’t have data persistence needs.
(31:35) K Scott asks about the ORM. Keith talks about why he built the query composer. Nodal’s ORM is called the composer, and it uses method chaining, then reduces everything down to one query. That gets around the n+1 issues you’ll run into with ORMs like Active Record. There’s a GraphQL example at graphql.nodal.com which can take a GraphQL query and translate it down to a single Postgres query. They’re looking at breaking the composer out to allow use in other GraphQL applications. K Scott asks more about the join syntax and lazy / eager loading.
What do you do for fun, music apps, what’s next?
(36:35) K Scott asks Keith what he does when he’s not working on stdlib and nodal. Keith says it’s the majority of his time now. Keith says he used to run.
(40:22) K Scott asks what’s coming up. Keith talks about the authentication and authorization layers as well as multi-language SDKs and support that are coming out soon. K Scott asks about the current auth story, and Keith says that currently you code it all yourself.
Deploying ASP.NET applications to Windows Containers with Docker
(00:30) Ben Hall gave a talk at NDC London in which he deployed Nerd Dinner and MVC Music Store 2.0 to Docker using Windows Containers.
(00:52) Jon says that Docker has been primarily Linux focused and asks how Windows fits in here. Ben explains the history of Docker and Windows Docker support on Windows Server.
(01:40) Jon asks how this works if you’re not developing on Windows Server.
(02:23) Jon asks why anyone would want to deploy Windows based applications using Docker. Ben talks about some advantages, including automation, tooling, and a standard approach to packaging and deploying applications – including applications that weren’t built with any thought of containers or automated deployment.
(04:30) Jon says that all the hype he’s seen related to ASP.NET on Docker are talking about ASP.NET Core. Ben talks about why non-Core apps on Docker are relevant.
(05:29) Jon asks how this compares to the traditional approach to just deploying using Hyper-V and full virtual machines. Ben describes some of the inefficiencies and just general heaviness around deploying an entire VM for an application, vs. lightweight container. Ben and Jon talk about some of the benefits, including deployment documentation as executable source code. Ben talks about the advantages of automating deployment of a set of resources using Docker Compose, as well.
(09:32) Jon asks about the different choices he’s got, including Windows Server Core and Windows Nano Server. Ben sorts him out.
(11:20) Jon asks how Hyper-V containers fit in. Ben talks about the security and isolation advantages due to having a separate kernel, especially when you’re dealing with a multi-tenant scenario.
(13:20) Jon asks about Ben’s recent experience with this due to deploying Katacoda. Ben describes how Katacoda, an online interactive learning platform for software developers. It leverages Docker to allow you to give you a terminal in a web page to let you start hacking and learning quickly.
(14:14) Jon asks about the business model for Katacoda. Ben explains that their main model is working with vendors, making it easier for their customers or potential customers to try out products with no install. They’re also working on versions for training, which eliminates the time and uncertainty of getting everyone’s machine configured.
(15:18) Jon asks what’s next, and Ben says the next big thing is Azure Container service – using Kubernetes to configure clusters across operating systems, optimizing for cost, etc.
What do you do for fun?
(16:20) Jon asks what Ben does for fun, and Ben reminds him that he runs a startup.
(00:30) Jon asks Rob about his presentation at NDC London. Rob’s talk started by describing how he got fired from a job by trying to do something that was NP-Hard. This past year he dug into understand complexity theory, mostly from the point of view of just recognizing the pitfalls. He once wrote a co-occurrence query for just two products (two products that are bought together frequently), and that worked just fine. However, trying to write a co-occurrence query for three or four products doesn’t work because it’s exponentially hard.
(02:23) Jon asks about the different classes of problems. Rob explains the terms, starting with polynomial time (P) problems, then talking about exponential and factorial complexity.
(04:10) Jon talks about how Rob’s co-occurrence query was exponentially hard, but for just two products it worked fine. Rob continues with his example from his talk about finding the best place for a group of people to go – that’s NP-Hard. But if there are only two people, you can handle it. You can get into solving some harder problems using concurrency and throwing machines at the problem, but you should understand it.
(5:10) Rob explains how ideas like page rank fit in, by using authority as a heuristic. Heuristics can be use used for other problems, like the travelling salesman – they won’t give you the provably best solution, but they will reliably give you a very good answer.
(7:45) Jon asks about the difference between decisions and optimizations. Rob explains that decision problems are NP-Complete problems – if you can represent a problem as a long boolean statement, it’s a boolean satisfiability problem. He describes how optimization problems
(10:12) Jon asks about Rob’s recent book, The Impostor’s Handbook. Rob explains why he wrote it, and the current audio / video updates he’s making for it.
(11:40) Jon mentions how there’s a lot more to the book than complexity theory, and Rob explains how it’s all related – complexity theory, foundations of computing, lambda calculus, etc. Jon asks Rob why he likes lambda calculus so much, and Rob talks about a presentation he really liked by Jim Weirich in which he built a y combinator, and he talks about some examples from his book using a y combinator in ES6 to do things like fibbonaci series.
(14:00) Rob’s book, The Impostor’s Handbook, is available at bigmachine.io.
(02:00) Jon comments on the star power among the contributors to HT. Richard calls out Shawn Wildermuth’s contributions and how he’s been applying his version update experience from his coursework to the project. HT got its start as the example project for the Visual Studio 2015 launch.
(04:04) Jon remembers to ask Richard to explain what HT is: open source software for disaster relief organizations. Richard was motivated by the realization that it’s hard for software developers to donate their skills to charity because software comes with an ongoing maintenance cost.
(05:35) Scott asks for a description of what the software does. Richard says Humanitarian Toolbox is a collection of projects, and they’re initially focused on the allReady project. allReady started to help the Red Cross organize and coordinate smoke detector installation efforts to prevent home fire disasters. Software can help through things like mapping, mobile apps, and Twilio based notifications. Just the simple addition of reminder notifications before going out to install smoke detectors has raised their install rate from about 30% to about 80%.
(09:00) AllReady is an ASP.NET Core web application using some default Bootstrap theming, and could definitely use some designer help. They work with the Red Cross to provide domain expertise. They’ve had some field trials, but are just now rolling it out broadly to the field now.
(11:35) Scott says that it sounds like HT is a little different from the drive by pull request model that’s common in the open source world. Richard says that pull requests really should start as an issue and a discussion before the pull request. They’ve consciously grouped issues so they can be managed at hackathons as well as milestones for releases.
(13:12) Jon notes that many open source projects evolve a pull request at a time and often don’t have a clear high level architecture. Richard says they’ve put some effort into architecture and hosting, with the realization that they’ll probably be hosting and maintaining the applications. He says that it’s great to be able to work directly with folks like Dominick Baier for IdentityServer, Jon Skeet for NodaTime, etc.
(14:55) Scott asks about a point Richard had made in his keynote about all the IoT devices we’ve got, but not enough software to go around. Richard says he doesn’t want the disaster relief heroes spending money on software. They don’t understand the impact mobile and cloud can have on their work, and we can help them. He talks about the possibilities for crisis check-in and citizen disaster evaluation using things like social media for things like bridge damage evaluation. There’s so much to be done, the job requires prioritization and building things in a sustainable way.
(18:32) Scott asks about how people can get involved; Richard points to htbox.org.
(19:12) Jon doubles back to the interaction pattern Richard talked about earlier with issues leading to discussion, then pull requests. Richard also refers to the weekly hangouts, where discussion and collaboration also happen.
(20:39) Scott asks what kind of help they could use. Richard says they’ve got a lot of people working on the ASP.NET Core side of things, but need more mobile development help.
Gadgets and Idle Chatter
(22:00) Scott asks Richard what his latest gadgets are. Richard talks about his new Dell 43 inch 4K monitor.
(23:30) Jon asks about Richard’s office remodel project, including LED lighting.
(25:30) Scott asks Richard what he’s doing when he’s not working. Aside from running a charity, he likes to get off the grid an hike in the Himalayas.
Jon and Kevin talk to Gary Ewan Park and Mattias Karlsson about Cake, a cross platform build automation system with a C# DSL to do things like compiling code, copy files/folders, running unit tests, compress files and build NuGet packages.
(01:48) Jon asks about the integrations and CI support. Gary says that’s a key feature: the build you run on your development machine is going to be as close as possible, if not identical to, the build you’re going to run on your CI server. Right now, Cake is set up with at least 10 online CI servers, which has the added benefit of providing them with a ton of badges for their GitHub readme.
(03:15) Jon says that he cloned the getting started repo and ran a script and some magic happened… but what exactly was that magic doing? Mattias explains that the bootstrapper (either build.ps1 or build.sh) fetches Cake from NuGet, then it launches the build.cake file.
(04:07) Jon asks if PowerShell on Linux will have an impact on Cake, or if the build.sh is simple enough that they’ll just stay with that. Gary says that the bootstrappers are very lightweight and normally don’t need to be changed (although you can if you want), so there’s no real advantage to moving to PowerShell since people who are building on Mac and Linux will probably prefer to just run a shell script.
(05:17) Kevin flips it around and asks if it matters for Bash on Windows. Gary says they’ve got this one covered, too, as they’ve already tested out Cake on Bash on Windows. Mattias says the hard part is the platform specific dependencies, and Gary agrees, saying that Cake is just a wrapper around the tools – there’s an expectation that the tools are either installed or available as a NuGet package.
(06:55) Jon asks if tools are well segmented per-project. Mattias confirms that everything is project specific – you can share if you want, but the default is that you can just clone a repository and build without thinking about dependencies.
(07:50) Jon asks how Cake works with Docker. Mattias talks about the preconfigured Docker images for Cake, which make it easy to easily test builds, handle integration tests, etc. He’s currently testing against Nano Server. Nano Server is very stripped down requires you to install a lot of prerequisites for Cake, so it’s nice to have a preconfigured Docker image for testing. Mattias talks about how versioning with tagged containers help them with integration testing. Gary talk about how they’re using Docker with Bitbucket Pipelines to really speed up their Travis build tests.
(10:25) Jon asks what a Cake script is – is it just C# code? Gary describes some of the additional build DSL features they’ve added, but other than that it’s just standard C# code.
(13:30) Jon said that he opened the Cake file in Visual Studio code and saw there was an extension for syntax highlighting, and asks if there’s additional tools for IntelliSense. Gary talks about the current status of OmniSharp when running against Cake. Note that since this podcast was recorded, this has improved as explained in this blog post: How to debug a Cake file using Visual Studio Code.
(15:28) Jon asks about debugging a Cake script. Mattias says they’ve added a debug switch to the Cake exe, which waits for you to attach so you can debug. Jon says it sounds great to have debugging support for a build script. Mattias says it’s nice, but since your build scripts are often running on build server it’s also important to have good logging support.
(17:02) Jon asks what the logging support is. Mattias says there’s a built-in abstraction for logging as well as an exception handler, so to break the build you can just throw an exception. Gary say that the default is just to log to the build server like AppVeyor, but if you want to log to something like logstash you can. Mattias says that they log to standard output and standard error, and every build system integrates well with that.
.NET Core support, DI and Modules, Cross-platform Support
(18:06) Jon asks more about the .NET Core port. Mattias says they’re just about done, just working with integration tests. For the most part it’s pretty straightforward, but you can run into things like differences between kernel versions on Linux. The biggest issue has been waiting for dependencies to be available on .NET Core. In the past they’ve relied on Mono for Linux and Mac, and there are slight differences compared to the .NET Framework, so it will be good to be on Core CLR everywhere. (note: the port to .NET Core has since been completed).
(19:52) Jon says he notices they’re using Autofac. Mattias says they use it for dependency injection throughout the codebase, but they’ve replaced it in some places for things like their module system.
(20:33) Jon asks how the module system works. Mattias says you just add a module folder in your tools and implement some attributes on your interfaces that indicate what your module should replace. (22:17) Jon asks about any issues they ran into with the .NET Core cross-platform port. Mattias says there are a lot of dependencies you don’t think about – for instance nuget.exe isn’t available cross-platform or on Nano Server, since it only has the Core CLR. Gary says they made a conscious decision not to implement .NET Core early, so they avoided some of the early adopter pain that some other projects ran into.
(23:47) Jon asks if they’re doing anything specific to handle platform differences. Mattias says that there are a few IFDEFs, but for the most part issues are around tools support, in which case it just won’t launch at all. Gary talks about platform specific criteria you can use in your build scripts to make platform specific decisions.
Cake Tasks, Parallel Tasks, Build System Integration, Unit Tests
(26:36) Jon asks how task names are used in a script. Mattias says that those labels are used to determine task dependencies. Gary says you can also use the labels as entry points which are specific to the build server.
(28:22) Jon asks if tasks are run in parallel. Mattias says it’s not currently multithreaded because logs would be difficult to follow. When you define a task, it’s just added to the graph, and nothing’s actually executed until you call RunTarget. Gary said there is an open issue to allow parallel execution of tasks, but they’re still working on a good story around debugging and logging. Mattias says it’s standard C# so there’s noting stopping you from running async tasks using the standard C# async methods.
(31:46) Jon says he sees support for several build systems and asks about how you integrate with them. Mattias explains how CI systems can call commandline options, and Gary says that all build systems have provisions for environment variables, and Cake provides a typed wrapper around that, so you get a strongly typed object from the build-specific provider to make informed decisions. For instance, when running on AppVeyor their script pushes the NuGet packages to the prerelease MyGet feed from the development branch and the official NuGet feed when running on release.
(34:09) Jon asks about assembly info patching and versions. Cake can update based on version info. It’s got file hash support, which is important for pushing to Homebrew.
(34:55) Jon asks about unit test integration. Gary says they have method aliases for calling common unit test frameworks, and you can easily extend for other frameworks. Unit test harnesses have published return values, so you can make informed decisions based on specific unit test results.
Publishing, Installers, Release Notes
(36:46) Jon asks about publishing options. Cake can build the nuspec for you, you can use your own if you want, or on Core CLR you can use dotnet pack. Jon asks about some of the different ways NuGet is used as a deployment package. (38:42) Jon asks about other installers like WiX, NSIS, etc. Gary says that Cake is really just wrapping tools with strongly typed classes, which makes it easier to pass in property values to tools. There’s no real magic, it’s just bringing it up an abstraction layer, so you don’t have to remember the command semantics of each tool. Jon asks if, in addition to simplifying how you interact with tools, the abstraction makes it a little easier to switch between tools. Gary says that’s true, and describes how this works in practice.
(41:44) Jon asks about the release note parsing feature. Gary says there’s an included release notes parser, which can extract version numbers and bulleted release features from a markdown file assuming it’s in a known format. This allows you to stamp your assembly with the version number, and use the bulleted release notes in your NuGet or Chocolatey release notes. There are also Git Release Notes and Git Release Manager which can be used to generate release notes from issues in a milestone based release on GitHub. Gary says they use this with Cake releases, so just tagging a release gives them an end to end publish process which sets version numbers and writes release notes for all their release endpoints (NuGet, Chocolatey, GitHub release, etc.).
(44:54) Jon asks about the social network support. Gary and Mattias explain how the different addins can announce builds to Twitter, Slack, Gitter, Hipchat, etc. They describe how things like this that are useful but not essential are available as addins.
(46:28) Kevin asks if there are any addins that they see as missing. Gary says there aren’t at the moment. He’s got a generic build script for all of his addins, so adding to the script rolls out to all of his addins.
(48:08) Jon asks about the discussion in their repo: How do we prevent Addin’s becoming stale? Gary talks about the problem (a community member creates a useful addin, then stops maintaining the addin). Their plan is to set up a cake-contrib organization which can help with long-term ownership and support for community addins, allowing them to push NuGet updates as required if the original creator is no longer available or involved. The plan is to ask users to move their addins into the new cake-contrib organization and make the cake contrib user a co-maintainer. Note: Since this was recorded, they’ve set up this organization, as discussed in the wrap-up after the show.
Open Source Project Case Studies: NancyFx and IdentityServer
(53:14) Jon asks about some of the open source project build conversions to Cake, starting with NancyFx. Mattias explains some of the challenges, as well as the clear payoff: build script pull requests very soon after the conversion. Jon asks some nitpicky details about why the NancyFx build script script is spawning a process instead of using a Cake wrapper. Mattias explains the process by which build scripts are refined, and Gary points out the advantage of always being able to spawn a process if you need to do something Cake doesn’t yet support.
(57:43) Jon asks about the IdentityServer migration. Gary said the IdentityServer team had already done the .NET Core port running under PSake and just wanted to migrate to Cake to get cross-platform builds going. It’s also a lot simple of a build process because it’s doing less.
(58:50) Jon asks about ilmerge suport. Mattias says it’s pretty popular for tools, to allow you to distribute the tool as a single exe. Gary says he uses Fody Costura instead to handle assembly merging. Jon says he doesn’t think any assembly merging support is available yet for .NET Core. Gary agrees, and Mattias says that .NET Native support will be helpful when it arrives.
Future Plans and Getting In Touch
(1:00:08) Jon asks about future milestones and what’s on the horizon after the .NET Core release. Gary hints at some upcoming releases, which are covered in the post-show wrap-up.
(1:01:20) Gary talks about some upcoming speaking engagements, and Mattias mentions the Gitter chat, Twitter and GitHub issues as ways for users to ask any questions.
(1:03:10) Cake Contributions organization The Cake Contributions organization (cake-contrib on GitHub) is now in full motion. It’s a common place where people can put their Cake addins & modules, if they want. They still maintain and for all purposes “own” the code, but they get access to resources like CI services, common build scripts, better exposure and the Cake core team can assist with merging pull requests, fixing issues and pushing to NuGet if addin grows stale or the author just too busy with life in general. It’s been well received and quite a few projects has moved over.
(1:03:52) 0.16 Release with .NET Core support So in addition to full .net Framework support, Cake now supports .NET Core (netstandard 1.6). There have been a few patch releases following this, so as of Oct 11 they’re up to 0.16.2. Also, due to adding .NET Core support, it’s now possible to debug a Cake file with Visual Studio Code. That’s cool, because there’s now cross platform debugging support for Cake. There’s a blog post showing how to do that.
(1:04:25) Frosting Frosting is a stand-alone .NET Core runner and host for cake. Cake uses the Roslyn scripting API and provides a DSL for fetching dependencies like tools and addins. Frosting uses the .NET core SDK to handle dependencies, and your cake build script is really just a .NET Core console app. Both are running against the same Cake.Core and Cake.Common packages, they’re just being hosted differently. This gives you some advantages – you’ve got full Visual Studio IntelliSense and tooling support, use the standard methods to package your build as a NuGet package, etc. Frosting also includes a dotnet CLI tool, so you can call dotnet cake.
(1:05:46) Cake for Yeoman support This adds a cake generator which makes it easy to bootstrap Cake, including a build script, bootstrapper scripts and config files in the current directory just by typing yo cake. There’s also scaffolding support to create a new .NET Core project using Frosting with yeoman.
(1:06:18) Cake for Visual Studio This is the first release of the Cake for Visual Studio extension. It includes syntax highlighting, Task Runner Explorer integration, bootstrapper commands in the Build menu, project templates for Cake addins, Cake Addin unit test projects, and Cake Modules
(02:55) Jon asks Matt to overview React. Matt explains how React handles views, components, and unidirectional databinding. Matt says it’s nice that it moves us towards the component model in the HTML world – moving away from the jQuery direct DOM manipulation approach to a more holistic approach.
(05:40) Jon says that he thinks that components introduce some problems of their own and asks Matt how React avoids that complexity. Matt says that’s why Flux and Redux were introduced.
(06:35) Jon asks what React Native is. Eric explains that React Native allows you to take the same concepts and tool used for building React web applications and use them to build native applications. It started at Facebook and has picked up a lot open source momentum.
How React Native compares to Cordova, Xamarin, etc.
(07:33) Jon says there’s always heated discussion when you talk about building native applications using web technology. What’s the overhead? Is this similar to Cordova? Eric explains the differences.
(11:22) Jon asks where React Native fits in on the scale from Cordova to more native tools like Xamarin.
Technical Implementation: How React Native creates Windows UWP applications
(17:30) Jon asks about how elements are styled, since from his experience that’s all done using CSS styles. Eric says they use a subset of CSS called Flexbox using a library by a developer from Facebook. Matt says when they ported over the F8 app to Windows, they leveraged the same styles and platform specific style pattern that had been used for iOS and Android.
(19:23) Jon asks what was most difficult in moving the F8 application to Windows. Eric says the styling part was pretty easy. The hardest part was in handling platform specific support. Matt describes an issue with linear gradient support.
Nuts and bolts: Data, Extensibility, X-Plat Dev, Building and Publishing, Scaffolding, Tools, Testing, etc.
(21:55) Jon asks about any other platform specific issues. Eric says that it’s conceptually similar to Xamarin in that are you need to keep in mind that you’re writing to a native front end experience. He explains that it’s a happy medium between web front-end (Cordova) and native in that the simple UI components like text are easily shared, but you can target more platform specific controls as needed. He talks about some differences in things like menus, split views and carousels.
(23:50) Jon asks about the cross-platform development experience? Do you use a lot of emulators? Eric says he mostly works on Windows using the Visual Studio Android Emulator. He says when he’s adding a new feature to React Native for Windows, he’ll pop open the Android emulator and use the UI explorer. Matt says he also uses Android Studio and Xcode.
(25:44) Jon asks if there’s a long build step, or if there’s a quick feedback cycle. Eric says that React is built to make the developer experience really fast, and explains how hot module reloading makes it possible to see application code changes without reloading or even losing current running state. Matt says that you also get the full Chrome debugging experience.
(27:15) Jon asks how you package your application for publishing. Matt talks about the publishing steps.
(28:10) Jon asks about the rnpm (React Native Package Manager) command-line tool. Eric explains how rnpm helps you include dependencies in your React Native applications. It’s extensible, so Eric and Matt used that to implement the Windows scaffolding experience.
(29:52) Jon says that in the scaffolded application he got when he ran rnpm, there are OS specific files for each view and asks if it’s possible to use a consolidated view for all platforms when appropriate. Eric says you’ll generally need a platform-specific entry point for each OS, but once you get past that you can do quite a bit with the core React Native elements.
(31:15) Jon asks if it’s possible to create additional platform-specific controls. Matt talks about support for native modules (e.g. dialogs) and native components (e.g. ImageView).
(33:20) Jon asks how he actually runs the application as he’s developing it. Matt says they just read in from the command-line to start the packager and build and run the application using Visual Studio. The docs currently say Visual Studio is required, although that isn’t necessarily the case if you kick off your build using msbuild.
(34:30) Jon asks if it could be possible to create a React Native Windows application on Mac or Linux, or if you’ll always need Windows due to the SDK. Matt says you’ll always need the SDKs, but Eric points out that there is a web-based React Native Playground (rnplay.org), and they’d like to reach out to them to get Windows support added.
(35:33) Jon asks if anyone is using it beyond the F8 application. Matt says not yet, they’re still working on the dev experience before they start heavily onboarding people. Jon asks who’s using React Native in general and Matt points out the React Native Showcase.
(36:50) Since part of the idea of UWP is that it runs on multiple Windows platforms, Jon asks if they’ve done anything with that. Eric talks about how part of their F8 demonstration included showing the application on Xbox, mobile devices with Continuum and desktop. They’re interested in looking into HoloLens and IoT as well.
(38:00) Jon asks if it’d be possible to leverage React web code in a React Native application. Eric says that at F8 he saw a talk on the possibility of bringing React Native to web. There are definite sharing points in code between web applications and React Native apps, although you should expect to write native views, and points out that there is work going on to bring React Native to Tizen and macOS.
(39:35) Jon asks if React Native for Windows is a science project or an ongoing project. Eric and Matt say they’ll be watching for community interest. They expect to keep up with React Native releases, and are hoping for community contributions. Eric says there are hundreds of React Native community modules, and they can’t scale to supporting all of them without some community involvement.
(01:58) Jon notes that they’re using Windows and CentOS and asks why CentOS as opposed to other Linux flavors. Nick says they tried Ubuntu first, but it’s more tuned for clients. CentOS is a variant of Red Hat Enterprise Linux, so all the packages work.
(02:53) Nick says that they use whatever’s the best tool for each job – factoring in the costs of each new tool (training, migrating, supporting, vendor overhead, etc.). They run Elastic Search, Redis, HAProxy and Logstash.
(03:50) Jon asks about how they’re using protobuf to serialize the information they’re persisting Redis. Nick talks about some specifics, including the different levels of caching that they’re doing. They’re using pub / sub with Redis via websockets. They’re not clustering, partly because there’s one Redis cache per SE site. He and Jon discuss how multitenant scenarios often require custom implementations.
(07:20) Jon asks about how they’re using websockets. Nick says that they’re used in a lot of places for optional updates. Their problems come from running on very few servers – they end up with huge connection tables per server.
(08:23) Jon asks about one of their Redis instances that’s handling machine learning with Providence. Nick says they log some metadata and performance information about about every single request. Providence is a system their data team wrote that analyzes the data, figures out locations, suggested user tags, etc. As a user, you have control over the personalization and can download the data or disable recommendations if you want. There’s also a mobile feed, for mobile apps as well. There are 40k ops/sec all day long against Redis. Scott K asks if he can manipulate his feed.
(11:51) Scott K asks about the L1 and L2 cache that Nick’s talked about. Nick clarifies that he’s been referring to HTTP caching on the web server and Redis caching.
(19:48) Jon asks about the Elastic Search implementation. Elastic Search doesn’t really support types, it has field groupings, which makes the upgrade more difficult. Nick explains that things are pretty vanilla now, but they’d like to make some customizations to support nested search results when time permits.
Data and SQL Server
(21:03) Jon asks about their SQL Server implementation. Nick talks about the clustering setup.
(21:49) Jon asks what version of SQL Server they’re using. Nick says they’re currently running the latest version of SQL Server 2014 and will move to SQL Server 2016 as soon as it releases. [Note: SQL Server 2016 has since released and they’ve upgraded.]
(22:11) Nick talks about some of the top reasons they’re looking forward to SQL Server 2016: string_split and JSON parsing. These are both useful for queries that take a list as a parameter. Jon reminisces about a time long ago when he used XML to pass lists to SQL queries.
(23:32) Kevin asks if they’re able to do a piecemeal migration without downtime. Nick explains how they do upgrades using replicas. They can test on other replicas, then fail over to them, or roll back to the previous master. They hate Windows clustering, and Windows Server 2016 and SQL Server 2016 should soon support distributed affinity groups which would allow them to do simple affinity group based upgrades.
(26:15) Jon asks about SQL Server on Linux. Nick says he can’t really talk about it.
(26:35) Jon asks if they use SQL Server Hekaton / In-Memory OLTP. Nick says they don’t, they run enough memory in their database servers that it’s not needed. Nick says it’s more for high-frequency no-lock access.
(29:27) Jon asks about how they handle migrations. Nick explains how changing tables is handled via a migration file and a ten line bot.
Source Control, Localization, Build
(30:47) Jon asks about their use of GitLab. Nick says it works okay, but they’re testing GitHub Enterprise internally due to performance. GitHub is significantly faster, search works a lot better (due to using Postgres search rather than Elastic Search), and there are some nice new features in GitHub like squashing commits.
(32:23) Jon asks about the localization features and is educated about the ja, ru, pt and es versions of stackoverflow. Nick explains some of the different localization issues that you run into in localization. Most localization solution work by string replacement, which requires string allocations. That doesn’t work at scale. They’ve written a system called Moonspeak which uses Roslyn to precompile view. This allows a direct response.write of the localized string via switch statements, which is a lot more efficient. They haven’t had time to open source it yet, but would like to.
(36:36) Jon asks about their build process using MSBuild. They use it mostly because it’s what the tooling uses. They could customize it more with PowerShell, but that would tie them more to TeamCity and Nick’s not sure there’s a benefit to making that move. Nick’s waiting to see where csproj is going – he’s got some big doubts it’ll be as terse as project.json, but he’s interested to see. Nick says historically MSBuild has been optimized for three-way merge generation; Jon says that was technically Visual Studio’s fault since MSBuild actually has had glob support for a while.
Upcoming Technologies, Visual Studio
(40:35) Nick complains about how slow Visual Studio is to reload projects. Their developers have scripts that just kill and restart Visual Studio, because that’s faster than handling project reloads.
(41:16) Jon asks if Nick’s played with Visual Studio “15”. Nick wonders about the technology used in the installer. He says it’s generally good, but they’re running into some issues with solution files changing when moving between versions.
(42:33) Nick says that they generally don’t ever use File / New, they copy an existing project and rename things. There’s a discussion about whether it’s possible to customize project templates. Jon says you can export a project as a solution template; K Scott mentions that SideWaffle has some capabilities there, too, but there was some “wonkiness”. And what’s the deal with GUIDs in project and solution files?
(46:09) Scott K mentions a command line base project generator that he started on years ago called ProjectStarter. He wishes that it was possible to configure Visual Studio to define a custom build tool rather than assuming everything’s in csproj. He gets that Visual Studio features like IntelliSense depend on controlling the build, but doesn’t like that Visual Studio has to “know everything about everything”.
(48:55) Jon says he sees two ways that cross-platform can work: either make the frameworks able to work without the tools knowing and controlling everything, or updating Visual Studio Code so it’s able to know and control everything. He hopes it’s the first way.
(49:50) Nick complains about how sometimes in-memory builds don’t reflect changes, or csproj doesn’t save before a build. He’d like everything to save before builds. Scott K calls out the Save All The Time extension for Visual Studio that Paul Betts made.
(50:40) Jon asks Nick if they’ve looked at ASP.NET Core. Nick says that they’ll mostly be starting with their internal tools. They have several libs that they’ll need to port, and they’ve got some difficult problems with libraries like MiniProfiler that need to support both .NET 4.x and .NET Core because the underlying APIs have some significant differences. You can’t just multi-target code that targets things like HttpContext. Other libraries like dapper and stackexchange-redis haven’t been as bad, and they’ve been working on them because lots of other developers are depending on them.
(55:03) Jon calls out some of Nick’s recent C# 6 tweets. Nick says he likes null coalescing and ternaries – they see more terse code as a lot more readable, but it of course varies by team. Roslyn has been really big for them, things like Moonspeak rely on it.
Questions from Twitter
(56:23) Matt Warren asks “What performance issues have you had the most fun finding and fixing.” Nick mentions the tag engine and a fun debugging issue they ran into where the TimeSpan int constructor uses Ticks rather than seconds or milliseconds, so their cache code was only caching values for tiny fractions of a second rather than thirty seconds. They find out so many issues using MiniProfiler; he wishes more developers would use MiniProfiler (or another tool like Glimpse) in their applications. They run MiniProfiler for every single request on StackOverflow and the overhead is minimal – if they can do it, you can do it.
(58:17) Matt Warren asks “What the craziest thing they’ve done to increase performance.” Nick talks about the IL related work they’ve done – sometimes instead of conditional code, it’s faster to just swap out the method body. They’re pragmatic, they only do this for extreme cases like things that run for every request and have real performance implications. What’s the trick for StackOverflow? Keep it simple.
(00:44) Jon mentions the Bash on Windows announcement at Build and asks if Kevin or K Scott have played with it. This devolves into a discussion of Windows Insider previews. Jon likes it and talks about the steps for enabling Windows Insider preview builds. K Scott has been scared to try it, but it sounds like he’s convinced. Kevin is put off by the Insider term – Windows Insider, Visual Studio Code Insider previews, etc. K Scott adds “Windows Insider” it to his e-mail signature.
(05:15) Jon talks about the steps for enabling Bash on Windows. or Windows Subsystem for Linux (Beta), to use the official terminology.
(07:08) Feel the excitement of listening to someone type commands into a console window as Kevin asks questions about what’s installed and Jon tries to apt-get it all. K Scott and Kevin wonder about how things like filesystem and processes work, and Jon tries to make up answers.
(09:50) Kevin says it feels like an admission of defeat to add *nix support to Windows. Jon says it feels practical to him – developers are building for multiple operating systems (especially including mobile), so it’s nice to have it supported.
(11:19) Kevin’s ready for Cygwin to die in a fire, and Jon’s excited about ssh working less horribly on Windows. Kevin says the race is on to get Wine working on Bash on Windows.
Angular 2 and React
(12:55) Jon worked on a hands on lab for Build that had master-details using Angular 2 and ASP.NET Core. He said Angular 2 seemed a lot simpler than Angular 1 now. K Scott said the component model is simpler, but he’s seeing some resistance to the ECMAScript / TypeScript updates, new binding syntax, etc. The Angular 1.5 release also includes a component model that’s a much easier programming model. It almost feels like some older Microsoft component-based programming frameworks going back to Visual Basic 6: you’re working with components that have simple properties and events. The guys speculate on how soon someone will build the big visual editor for Angular 2.
(18:50) The guys discuss how Angular 2 and React mindshare will play out. Jon likes React as long as he never views source. Jon thinks the unidirectional flow is really simple, and Kevin agrees – after years of lower level Backbone, the simpler flow in React saves some mental energy.
(20:29) Jon mentions that React Native recently came to Windows, too.
Devices: Surface Book, Kindle and big batteries
(20:56) K Scott got a new Surface Book (after waiting to make sure nothing new was coming out at Build). He says it’s the best piece of PC hardware he’s bought in years – the build quality is good, the keyboard is good, he gives the trackpad of 9 out of 10. He says that the detachable tablet is a bit large as a tablet, so he’s using an older Surface for reading.
(23:58) Jon jokes that K Scott’s not likely to buy a Kindle and says he gradually stopped using his Kindle when he moved to Audiobooks, and kind of associates reading with work now. Kevin says that’s sad. They talk about Kindles for kids’ books.
(27:07) Kevin says the two big things he picked up for the new Kindle are physical page buttons and a three month battery. He says the main thing he likes about Kindle for both him and his kids (as opposed to a tablet) is that it forces you to read instead of getting distracted. The guys decide that tripling the life of an already one month battery isn’t a huge win.
(29:20) Jon says he recently bought a portable battery that can recharge his laptop, which is handy for long flights. (note: he said it was iPad size, it’s a lot smaller but is 1.2 pounds)
(31:15) Jon asks Kevin about new Apple hardware. Kevin says the iPad Pro screen is apparently astounding – he’s expecting them to be amazing in a few generations. Same for Apple Watch – he doesn’t have one yet, he’s waiting for version two. Jon says the improvement from Microsoft Band 1 to Band 2 was pretty nice, especially in the industrial design.
Lightning Round: Will Windows Phone be dead in one year?
(33:40) K Scott asks if Windows Phone will be dead in a year. Jon hopes not, as he just bought a Lumia 950 XL. He’d had a budget Blu Windows Phone since September, and the Windows 10 Insider builds were nice, but the camera wasn’t very good. He really likes Windows 10 as a phone operating system and thinks it’s sad that so few people will actually see it.
(35:55) K Scott got a Lumia 950 XL in January (when he dropped something on his previous phone). He got the docking station, too, and said it worked surprisingly well. The guys discuss how useful docking a phone is; Jon postulates that it could be useful for someone who does everything on their phone and occasionally needs to write a long email or edit their resume – especially if it can be hooked up to a TV.
(38:30) Kevin says that Windows Phone isn’t dead. it’s undead. It will linger on in a zombie-like state indefinitely.
Scott Koon sends us out with a request for further information by e-mail.
(00:54) JG asks Jon if he showed off any scary hacks. Jon describes an attack in which they made an executable editable both on disk and in memory, then edited both the IL and assembly code to do things like inject direct database attacks (bypassing . He describes how that could be defended using enterprise defensible architecture by talking to databases through services which can implement security layers. The goal is to prevent an attack at one layer from moving through the rest of the enterprise.
(1:59) JG says that often hears that major hacks occur by web application attacks that are then escalated to database attacks, often through password reuse. Jon says that’s true, since web applications are often deployed to the same server as a database or authentication server. He recommends using a service that’s locked to a single port, with security unit tests.
(2:43) JG asks which patterns he’s describing are unique to .NET development. Jon says that he’s emphasizing patterns that are easy on .NET – for example, REST services are easy to implement on .NET as compared to C++. He’s advocating architectural changes that are relatively easy to implement in .NET applications provided you start with them early on (rather than trying to retrofit security later).
A specific example: Protecting a medical record
(3:33) JG asks for some specific examples. Jon says they talked about security unit tests and user stories and gives and example from his talk about a medical record that’s being sent through an enterprise securely. To do that, you’d need to encrypt it on an edge node, so the web server and database don’t have decrypted data or decryption keys. Instead, you use a key server on a segmented network. Because of this, at no point could a sysadmin have gotten access to the record because it was encrypted at all steps.
(4:49) JG points out that Jon is talking about preventing access to sensitive data by sysadmins. Jon says that you should consider your attackers to have more power than a sysadmin – they refer to attacker privileges as "above admin" because they’ve taken over your AD infrastructure, passwords on routers, dropping new firmware, etc.
(5:50) JG refers to James Mickens’ keynote from the previous night (Herding Code interview here) and asks if there’s any hope. Jon says you need to plan, you need to minimize pivot points to prevent an attacker from moving between servers. He discusses potentially using static HTML that calls into secured REST services rather than web applications with direct database access, because building security around a REST service is much easier than securing a web application and web server.
Mobile and desktop application security
(9:03) JG asks about mobile application security, since he frequently hears about mobile apps that are supported by unsecured (or very poorly secured) backend services. Jon refers to Amazon and Twitter as examples of companies with published patterns for secured backend services.
(9:52) JG asks if for some tips on how different systems on the network should be secured, referring to desktop applications. Jon says that each system should defend itself from the other systems, so in this case the other systems should assume that the desktops could be compromised, the desktop applications should assumed the database can be compromised, there needs to be thought about defending the outgoing APIs, etc. There needs to be a plan for how to take things down and respond.
What do you wish you’d done?
(10:35) As a thought exercise, assume that your database or web server has been compromised: what do you wish you’d have done? Do that for different pieces of your application architecture. Jon says that you can make it a fun exercise, pit dev teams against each other, etc. Security should be fun and easy if you’re doing it right.
Wave 3D: A 3D operating system front end
(12:07) JG asks Jon about the 3D operating system he’d mentioned before the call. Jon describes Wave 3D, a cross platform front-end to multiple operating systems (Windows, Mac, Linux, Android) that gives the same exact experience for all backend systems. It can be connected to Amazon, Google and Dropbox document storage.
(12:56) Jon says it’s running on Mono and Unity and says it compares pretty well with 3D operating systems in the movies, and they’re looking at launching it via Kickstarter. It provides a simple, install-free environment with a document viewer, movie player and more on every platform instantly.