The Phoenix Files

Using AI to Boost Accessibility and SEO

2024-06-24T00:00:00+00:00

This article is about putting AI to work for us by making our websites more accessible and perform better with SEO. Fly is an excellent platform for running services that prioritize accessibility and SEO. Get started now!

In the fast-paced world of digital content, the adage “a picture is worth a thousand words” remains as true as ever. This is the case for Sarah, a content manager at a popular online magazine. She is racing to finalize a feature article called “The Hidden Gems of Urban Street Art,” a visually rich showcase of Toronto’s vibrant but oft-overlooked urban art scene. As the deadline looms near, she faces a dilemma: how do we ensure that these compelling images are not only eye-catching but also accessible to all readers and at the same time optimized for SEO?

Here’s an example photo from Sarah’s feature article that needs a description for accessibility and SEO:

Photo by Scott Webb on Unsplash

We’ve been tasked with solving this recurring need, the need for context-aware alt tag descriptions and caption text for images that accompany an article. “Context-aware” means that the description should relate to how the image is being used and not simply a description of the image. With these two versions of an image description, we improve both SEO (Search Engine Optimization) and the accessibility of our content.

We’re turning to AI to help automate the process and bulk process hundreds or thousands of past images as well.

Our job is to use AI to create decent alt tag and caption text on hundreds or thousands of images.

For the AI, we will leverage both OpenAI’s ChatGPT and Anthropic’s Claude LLMs (Large Language Models) through the Elixir LangChain library to perform AI analysis on images. The same process can be applied using other languages and tools.

Let’s first understand more about what makes high quality image description text so we know what we’re aiming for.

Context matters

How the image is being used matters. The usage provides additional context for how the images should be described. Let’s consider the following image when used in different contexts.

Photo by Aaron Andrew Ang on Unsplash

Description with no context: Elderly man sitting on a bench by a lakeside, reading a book with mountains in the background.

Description in the context of a story about a local author: René Favre reading beside a serene lake with misty mountains in the background.

Description in the context of a travel destination: Older man sitting on a bench overlooking Lake Lucerne with mountains in the background.

How the image is used changes how it should be described. This impacts what aspects of the photo should be called out as relevant or ignored entirely.

Okay, so the description needs to be in the context of our article. What are the current best practices around alt text and captions?

What is alt text?

Alt text, or alternative text, describes images in HTML code. It’s essential for two main reasons:

Accessibility: Screen readers use alt text to convey image content to visually impaired users, ensuring that everyone can fully engage with the webpage.
SEO: Search engines read alt text to understand and index images, enhancing webpage visibility in search results.

In a nutshell, alt text makes images accessible and boosts their discoverability, benefiting both users and search engines. When we make our content more accessible to users and to search engines, it benefits us.

What are captions?

The <figure> and <figcaption> are semantic HTML5 elements that work together to provide context and captions for images, diagrams, or other media within a webpage.

ARIA (Accessible Rich Internet Applications) tags are special attributes added to HTML elements to enhance web accessibility.

The Mozilla documentation encourages us to semantic HTML elements over ARIA tags whenever possible.

If at all possible, you should use the appropriate semantic HTML elements to mark up a figure and its caption — <figure> and <figcaption>.

In HTML, it looks like this:

    <figure>
  <img src="image.png" alt="put image description here" />
  <figcaption>Figure 1: The caption</figcaption>
</figure>

  

On the webpage, the default styling places the caption below the media, although this can be modified through CSS.

We have more freedom with the content displayed in a figcaption, the only rule is there can only be one per figure. It is self contained with the media it describes, making it more portable within a page. A general recommendation is to connect the media to the page, providing context for why it is there. Lastly, keep in mind the visual display of the figcaption’s contents and make the text concise.

In a nutshell, <figure> and <figcaption> provide visible captions and context to sighted readers, assistive technology, and search engines, effectively connecting media with the surrounding article or content on the page.

How can AI help?

AI can analyze an image and provide a text description. Recent model improvements from both OpenAI and Anthropic make this much easier and more powerful.

However, unless we want the AI to run amok and wax poetic on one image and do something completely different on the next, we need to provide the set of constraints and instructions that will get us what we want. In other words, prompt engineering!

To get consistent, well performing text, we need to understand a bit more about what makes for good alt text so the AI can give us exactly what we need.

This is a list of the relevant recommendations for alt text.

Concise Description: Keep the ALT text concise and to the point. Aim for a simple sentence or a fragment that conveys the essential information about the image. Typically, 125 characters or less is advisable as some screen readers may truncate longer text.
Content Relevance: Focus on the information that the image conveys and its context within the page. Describe what is relevant to the content and functionality of the site. If the image contains text, such as a logo, include the text in the ALT tag.
Avoid Redundancy: Do not include phrases like “image of” or “graphic of,” as screen readers often provide this context to their users. This can be redundant and unnecessarily verbose.
Context Matters: Tailor the ALT text depending on where and how the image is used. The same image might need different ALT text in different contexts.
We won’t go into Localization here, but AI can help with this too!
Localization: When dealing with multilingual sites, make sure to translate the ALT text appropriately, keeping the same considerations in mind as for the original language.
Use Proper Grammar: Even though it’s brief, ensure your ALT text uses correct spelling, grammar, and punctuation. This improves clarity and the overall user experience.

AI can help with all of these points! This dramatically simplifies the task, especially when we need to provide descriptive text for many, many images.

As for the figcaption tag, we have more freedom with that and we’ll try to pull in more context from the article. It’ll be fun!

Bring on the AI!

We understand that context matters and we have guidelines for how the alt text should be generated. We’re ready to write code!

To send an image to both OpenAI and Anthropic, we’ll base64 encode the image and submit it that way. In Elixir, that process looks like this:

    image_data =
  image_path
  |> File.read!()
  |> Base.encode64()

  

Let’s start with OpenAI’s ChatGPT service. We need model gpt-4o for the image abilities we are using.

    alias LangChain.ChatModels.ChatOpenAI

openai_chat_model = ChatOpenAI.new!(%{model: "gpt-4o"})

Next, we’ll setup the messages we’re sending to the LLM. This is the “prompt engineering” step.

In the following code, notice that the “system” message provides the general context for what we are doing and what we want from the LLM.

The “user” message is made up of two parts:

PromptTemplate: supports variable replacement tags using EEx templates. This allows us to easily customize the prompt for each image as we process through a whole batch. This turns into a ContentPart.
ContentPart: Makes it easy for us to provide our image directly to the LLM.

We provide the LLM with the context for the task, specific instructions about an image, and an image to analyze with a “vision” enabled model so it can perform the task.

    alias LangChain.Message
alias LangChain.Message.ContentPart
alias LangChain.PromptTemplate

messages = [
  Message.new_system!("""
  You are an expert at providing an image description for assistive technology and SEO benefits.

  The image is included in an online article titled "The Hidden Gems of Urban Street Art."

  The article aims to showcase the vibrant and often overlooked artworks that adorn
  the nooks and crannies around the city of Toronto Canada.

  You generate text for two purposes:
  - an HTML img alt text
  - an HTML figure, figcaption text

  ## Alt text format
  Briefly describe the contents of the image where the context is focusing on the urban street art.
  Be concise and limit the description to 125 characters or less.

  Example alt text:
  > A vibrant phoenix graffiti with blazing orange, red, and gold colors on the side of a brick building in an urban setting.

  ## figcaption format
  Image caption descriptions should focus on the urban artwork, providing a description of the appearance,
  style, street address if available, and how it relates to the surroundings. Be concise.

  Example caption text:
  > A vibrant phoenix graffiti on a brick building at Queen St W and Spadina Ave. With wings outstretched, the mural's blazing oranges, reds, and golds contrast sharply against the red brick backdrop. Passersby pause to observe, integrating the artwork into the urban landscape.
  """),
  Message.new_user!([
    PromptTemplate.from_template!("""
    Provide the descriptions for the image. Incorporate relevant information from the following additional details if applicable:

    <%= @extra_image_info %>

    Output in the following JSON format:

    {
      "alt": "generated alt text",
      "caption": "generation caption text"
    }
    """),
    ContentPart.image!(image_data, media: :jpg, detail: "low")
  ])
]

  

NOTE: Make sure the :media option matches both the image and what is supported by the LLM you are working with.

NOTE: The use of a PromptTemplate and <%= @extra_image_info %> in the user Message. As we are processing a whole set of images, data from the system where we get the image is rendered into our prompt, helping to further customize the generated text from the LLM, making it far more specific and relevant.

JSON Output

For each image, we want two pieces of generated content. To make this easy on ourselves, we instruct the LLM to output the two pieces of information in a JSON object.

Specifically, we instruct it to output in the following format:

{
  "alt": "generated alt text",
  "caption": "generation caption text"
}

To make working with that output easier, we’ll use a JsonProcessor for processing messages from the LLM. It has the added ability to return a JSON formatting error to the LLM in the case when it get’s it wrong.

We’ll see this next when we put it all together.

Making the request

Everything is ready to make the request!

We have the image
We setup which LLM we are connecting with
We provide context in our prompt and instructions for the type of description we want

Now, we’ll submit the request to the server and review the response. For this example, the image_data_from_other_system is a substitute for a database call or other lookup for additional information we have on the image.

    alias LangChain.Chains.LLMChain
alias LangChain.MessageProcessors.JsonProcessor

# This data comes from an external data source per image.
# When we `apply_prompt_templates` below, the data is rendered into the template.
image_data_from_other_system = "image of urban art mural on underpass at 507 King St E"

{:ok, _updated_chain, response} =
  %{llm: openai_chat_model, verbose: true}
  |> LLMChain.new!()
  |> LLMChain.apply_prompt_templates(messages, %{extra_image_info: image_data_from_other_system})
  |> LLMChain.message_processors([JsonProcessor.new!()])
  |> LLMChain.run(mode: :until_success)

response.processed_content

  

Notice that when running the chain, we use the option mode: :until_success. Some LLMs are better are generating valid JSON than others. When we included the JsonProcessor, it parses the assistant’s content, converting it into an Elixir map. The converted data is stored on the message.processed_content.

If the LLM fails to give us valid JSON, the JsonProcessor generates a :user message reporting the issue for the LLM. The :until_success mode repeatedly makes the round-trip requests to the LLM, allowing it to correct any errors. But don’t worry, it won’t run forever! The LLMChain’s max_retry_count gives up after X failures, the default being 3.

Here’s a sample of what was generated when I ran it:

    %{
  "alt" => "Colorful mural of a face under a bridge at 507 King St E",

  "caption" => "A captivating face mural under the 507 King St E underpass, featuring vivid hues and expressive eyes. The art adds a pop of color to the urban landscape, drawing the attention of pedestrians as they pass by."
}

Great! We got highly specific image descriptions for both alt text and an use in a caption. Even more, it’s all written in the context of the article about urban art. Importantly, we got the data back in a machine-friendly format that our application can easily work with.

We’ve come this far, let’s see if it works with Anthropic’s Claude LLM!

Anthropic’s Claude

We’ll setup the Anthropic model to work with like this:

    alias LangChain.ChatModels.ChatAnthropic

anthropic_chat_model = ChatAnthropic.new!(%{model: "claude-3-opus-20240229"})

Make sure the model you select supports the “vision” ability. The one used here works just fine.

Our code looks exactly the same except we’ve swapped out the llm chat model to connect with.

    alias LangChain.Chains.LLMChain
alias LangChain.MessageProcessors.JsonProcessor

# This data comes from an external data source per image.
# When we `apply_prompt_templates` below, the data is rendered into the template.
image_data_from_other_system = "image of urban art mural on underpass at 507 King St E"

{:ok, _updated_chain, response} =
  %{llm: anthropic_chat_model, verbose: true}
  |> LLMChain.new!()
  |> LLMChain.apply_prompt_templates(messages, %{extra_image_info: image_data_from_other_system})
  |> LLMChain.message_processors([JsonProcessor.new!()])
  |> LLMChain.run(mode: :until_success)

response.processed_content

  

What do we get?

    %{
  "alt" => "Colorful street art mural with a face in vibrant colors on an underpass pillar in an urban setting.",

  "caption" => "A striking urban art mural adorns an underpass pillar at 507 King St E. The artwork features a mesmerizing face composed of vivid, rainbow-like hues and intricate patterns. Framed by the industrial yellow beams above, the mural transforms the concrete structure into a captivating focal point for passersby."
}

Nice! The Elixir LangChain library abstracted away the differences between the two services and with no code changes, we can make a similar request about the image from Anthropic’s Claude LLM as well!

Discussion

That’s a lot! Let’s recap the main points we learned from this.

Both OpenAI’s ChatGPT and Anthropic’s Claude models can interpret the meaning of images.
AI can create pretty good image descriptions that are context aware
AI can incorporate image specific descriptions to make the description hyper-specific
AI can return the text descriptions in multiple formats from the one request
AI can be used to batch process thousands of images if needed
AI libraries can abstract away the differences between services making our code less dependent on any particular service.

There are limits to what AI can do. AI won’t always perfectly interpret what’s in the image as well as a human. However, it can be done much faster, automated and performed on more images than is humanly possible. In many instances, that means we get good (if not perfect), context aware ALT text where we may otherwise have none because we don’t have the people nor the time to manually do the work.

Finally, this feature would likely end up being built into the Content Management Systems (CMS). In our fictitious scenario, that’s probably what we’d end up doing… building this feature into the company’s software. But now you’ve seen how to do it yourself! Sweet!

Final Thought

AI made our visual content more accessible to people using assistive technology and our image content will perform better in SEO.

Fly.io ❤️ Elixir

Fly.io is a great place to run your Phoenix apps. It’s easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Resources

The best practice recommendations we’re using come from these resources:

My Favorite new LiveView Feature

2024-05-21T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Phoenix LiveView v0.20.4 was released in February 2024. It included a new feature that hasn’t gotten enough recognition. This simple little feature is a nice improvement for developer experience. Where previously I used Alpine.js or needed to craft a custom hook, now it’s built-in!

What feature is it?

It is 🥁 drum-roll 🥁… JS.toggle_class/1

🥱 (yawn)

Don’t give me that! It’s actually really helpful. And it’s surprisingly versatile. Let’s take a closer look!

These examples all rely on Tailwind CSS classes that are being toggled.

Basic visibility

Let’s start with a basic example of toggling the visibility of an element.

The code for this is all in the DOM.

    <div
  phx-click={JS.toggle_class("hidden", to: "#toggled-content")}
  class="px-4 py-3 cursor-pointer bg-purple-100 rounded-md border border-purple-300 text-purple-600"
>
  <div class="font-medium">Visible content</div>
  <div id="toggled-content" class="hidden mt-2 text-purple-800 font-bold">
    Toggled content
  </div>
</div>

  

The phx-click uses JS.toggle_class/1 to toggle the hidden class on the DOM element with the #toggled-content id. In the example, the element starts off hidden. Clicking the element either removes the hidden class or adds it back.

But what if we want to toggle multiple items with the same click?

Toggle multiple elements at once

In this example, a single click toggles classes on two different items. Here’s what it looks like:

When the user clicks the “Click to reveal text” text, it is hidden and a previously hidden element is revealed.

    <div
  id="example-2"
  phx-click={
    JS.toggle_class("hidden",
      to: ["#example-2 > .title-content", "#example-2 > .expanded-content"]
    )
  }
  class="px-4 py-3 cursor-pointer bg-purple-100 rounded-md border border-purple-300 text-purple-600"
>
  <div class="title-content font-medium">Click to reveal the text</div>
  <div class="expanded-content hidden mt-2 text-purple-800 text-md font-light">
    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
  </div>
</div>

  

The JS.toggle_class/3 function takes a “to” option which supports a list of CSS selectors for selecting a DOM element to toggle the hidden class on.

Notice that we’ve added the .title-content and .expanded-content classes to the div elements solely for use as selectors. Clicking the element looks for an immediate child element with those classes and toggles the hidden class.

In the example, a single click adds the hidden class to one element and removes it from another at the same time.

Toggle with animations

We can toggle more than just the hidden class! Tailwind makes it easy to add animations based solely on classes. In this next example, we’ll animate a “down” chevron to rotate when clicked making it point “up”.

Here’s the code:

    <div
  id="example-3"
  phx-click={
    JS.toggle_class("hidden", to: "#example-3 > .expanded-content")
    |> JS.toggle_class("rotate-180", to: "#example-3 .chevron", time: 300)
  }
  class="px-4 py-3 cursor-pointer bg-purple-100 rounded-md border border-purple-300 text-purple-600"
>
  <div class="flex justify-between">
    <div class="font-medium">
      Section Title
    </div>
    <div class="chevron transition-all duration-300">
      <.icon name="hero-chevron-down" class="text-lg" />
    </div>
  </div>
  <div class="expanded-content hidden mt-2 text-purple-800 text-md font-light">
    <p>Cool extra content!</p>
    <p>Cool extra content!</p>
    <p>Cool extra content!</p>
  </div>
</div>

  

The code toggles the Tailwind CSS class rotate-180 from an item. It uses the ‘time’ option to express a 300 ms duration for the change.

The rotating item is a div with the chevron class, which includes other Tailwind CSS animation classes like transition-all and duration-300.

Clicking the element toggles the visibility and animates the “down” chevron to rotate and now point upwards, indicating that clicking it again will collapse the content.

Subtle, but effective. Most importantly, it was very easy.

Dynamic items in a list

All the examples to this point used a hard-coded DOM id. Frequently we’ll have lists of items that we want to support where we need a dynamically assigned id.

Let’s see an example of that next.

Typically, data like this would be loaded from a database. Here’s our list of items:

    items = [
  %{
    id: 1,
    title: "Famous Quote",
    details: "If you judge people, you have no time to love them. ~Mother Teresa"
  },
  %{id: 2, title: "UK Saying", details: "Keep calm and carry on. ~Winston Churchill"},
  %{id: 3, title: "I know...", details: "...more than I did yesterday."}
]

  

Here’s how we use the list of items in our template:

    <h2 class="mb-2 text-lg font-medium text-purple-900">
  Dynamic Items
</h2>
<ul class="grid grid-cols-1 gap-4">
  <li
    :for={item <- items}
    id={"item-#{item.id}"}
    class="px-4 py-3 cursor-pointer bg-purple-100 rounded-md border border-purple-300 text-purple-600"
    phx-click={JS.toggle_class("hidden", to: "#item-#{item.id} > .expanded-content")}
  >
    <div class="font-medium">
      <%= item.title %>
    </div>
    <div class="expanded-content hidden mt-2 text-purple-800 text-md font-light">
      <%= item.details %>
    </div>
  </li>
</ul>

  

The expression :for={item <- items} dynamically generates an <li> element for each item in the list. We use the item map to dynamically build a DOM id for the element: id={"item-#{item.id}"}.

The phx-click event uses the same approach to assemble the DOM id to find the element and toggle the hidden class.

Reusing the click event

When multiple elements in a component should all trigger the same event, duplicating a complex phx-click JS event can quickly become tedious. Fortunately, there’s a simple way to define the click event and share it among multiple elements.

In the view code, we can define a private function like this:

    defp expand_item(item_id) do
  JS.toggle_class("hidden", to: "#item-#{item_id} > .expanded-content")
  |> JS.toggle_class("rotate-180", to: "#item-#{item_id} .chevron", time: 300)
end

  

In our template, multiple elements can share the same phx-click event:

phx-click={expand_item(item)}

Defining the actions once and reusing them this way makes it easier to maintain when more complex interactions are needed.

Discussion

A major benefit to using JS.toggle_class/1 is avoiding the need to pull in another tool just because “it makes things easier.”

For simple visibility changes and animations like those demonstrated above, JS.toggle_class/1 is perfect! It’s simple, extensible, and best of all, it’s built-in!

Will this become a new favorite LiveView feature of yours?

Fly.io ❤️ Elixir

Fly.io is a great place to run your Phoenix LiveView apps. It’s easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

World Page Speed Test – planet-wide elastic scale with FLAME

2024-05-08T00:00:00+00:00

We’re Fly.io and we transmute containers into VMs, running them on our hardware around the world. We have fast booting VM’s, so why not take advantage of them?

I closed out the year the publishing a new library and programming model, called FLAME. If you missed it, FLAME replaces the need for AWS Lambda or Cloudflare Workers and the proprietary services that call proprietary services fractal of architecture patterns that come with them.

Instead, imagine if you could auto scale by wrapping any existing app code in a function and have that block of code run in a temporary copy of your app. That’s what FLAME is all about.

Since the release, I’ve built a few things using this pattern, and the most remarkable thing is just how unremarkable the solutions are. I can write my naive Elixir application on my laptop, deploy it to Fly.io running across the planet in a single command, then scale it out elastically by changing almost no code.

Measuring Page Speed

I’ve wanted to build a “World Page Speed Test” for a long time. Think Google’s Page Speed Insights, but as viewed from various observers around the globe. To do this correctly, you need to be running a full browser. And those browsers need to be running on servers around the planet. The client must download all scripts, styles, and images to reflect a real-world page load. Doing this at scale is usually Hard™. Doing this with Elixir, FLAME, and Fly.io is an afternoon of tinkering.

🔥🔥 Try World Page Speed Now! 🔥🔥

So we need a browser. Fortunately, the major browsers support headless drivers through a W3C standard. This allows starting a headless browser and communicating with it via local HTTP. You can drive page navigation, evaluate page JavaScript, and simulate user interaction.

Our Elixir app can start with the basics. We’ll need:

A process group of nodes and where they are located geographically
A LiveView page that accepts a URL
A running chromedriver process that can launch headless chrome sessions for us
The ability to tell chrome to visit a URL and monitor network performance events
When the page starts loading, we display a loading status
When the page finishes loading, we display the loading time

This doesn’t take much code. I’ll crib the highlights.

First, we need a way to locate all nodes on the cluster, and the Fly region they belong to. This lets us know where the chrome requests are happening from. The built-in Node.list() returns all reachable nodes, but we need metadata alongside their name showing where they are located.

Fortunately, we can use Phoenix.Tracker for this, which provides a process group with metadata:

    defmodule WPS.Members do
  use GenServer

  @tracker WPS.Tracker

  def list(group_name \\ __MODULE__) do
    Phoenix.Tracker.list(@tracker, group_name)
  end

  def start_link(opts) do
    GenServer.start_link(__MODULE__, opts)
  end

  def init(opts) do
    my_region = System.get_env("FLY_REGION") || "ord"
    group_name = Keyword.get(opts, :name, __MODULE__)

    {:ok, ref} =
      Phoenix.Tracker.track(@tracker, self(), group_name, my_region, %{
        node: Node.self(),
        machine_id: System.get_env("FLY_MACHINE_ID")
      })

    {:ok, %{group_name: group_name, ref: ref}}
  end
end

  

We define a WPS.Members module which starts a process and calls Phoenix.Tracker.track/5 to register the current node’s FLY_REGION.

Now we need a bit of HTML inside a LiveView which drives our headless chrome when the user hits “go”:

    def render(assigns) do
  ~H"""
  <h1>World Page Speed</h1>
  <form
    :if={!@ref}
    id="url-form"
    phx-change="validate"
    phx-submit="go"
  >...
  """
end

def handle_event("go", %{"url" => url}, socket) do
  validated_url = URI.to_string(uri)
  ref = make_ref()
  parent = self()

  node_times =
    for {region, meta} <- WPS.Members.list(), %{node: node} = meta, into: %{} do
      {node, Browser.Timing.build(validated_url, region)}
    end

  {:noreply,
  socket
  |> clear_flash()
  |> assign(ref: ref, uri: uri, form: to_form(%{"url" => validated_url}))
  |> stream(:timings, Map.values(node_times), reset: true)
  |> start_async(:timing, fn ->
    nodes = Enum.map(node_times, fn {node, _} -> node end)
    :erpc.multicall(nodes, fn -> timed_nav(node_times[node()], parent, ref) end)
  end)}
end

  

When the user hits the “go” button on the web page, our goal is to kick off some timed headless chrome navigations.

We’ll simulate a multi-node setup locally for now, with just a single member. We aren’t going to worry about going planet-wide multi-node just yet – though it won’t take any code changes to get there.

After validating the user’s URL, we build up a %Browser.Timing{} struct for reach member’s region in the cluster. Next, we asynchronously navigate to the page inside a start_async call.

Within the the start_async, we can see some built-in Erlang standard library treasures. :erpc.multicall accepts a list of nodes and a function to run. The Erlang VM will run the function on the passed nodes and blocked until it gets a result from all nodes. Any process id (Pid) we pass to the closure can just be messaged across the cluster as if it was local.

Processes on the Erlang VM are our messaging, concurrency, and state primitive. They’re used everywhere, and you can run millions per node. It’s a bit like if any object in your OO runtime had a globally addressable reference that allowed you to call methods on a given instance from anywhere in the cluster. And each object ran in its own lightweight, preemptable thread.

Our local node setup is the only node running in dev, so the function will just run locally. The timed_nav/3 function looks like this:

    def timed_nav(%Browser.Timing{} = timing, parent, ref) do
  timing = Browser.Timing.loading(timing)
  send(parent, {ref, {:loading, timing}})

  case Browser.time_navigation(timing, @browser_timeout) do
    {:ok, %Browser.Timing{} = timing} ->
      send(parent, {ref, {:complete, timing}})

    {:error, {reason, %Browser.Timing{} = timing}} ->
      send(parent, {ref, {:error, {reason, timing}}})

    {:error, reason} ->
      send(parent, {ref, {:error, {reason, Browser.Timing.error(timing)}}})
  end
end

  

All we do here is send a message to the parent process (our LiveView), which we’ll handle in a moment.

We first tell it we’re about to start loading. Next, we call into a Browser.time_navigation function, which asks headless chrome to navigate to the webpage and give us the timing details. The headless chromedriver glue isn’t interesting here, but you can check the source if you’re interested.

If the navigation is successful, we send a {:complete, timing} message to the LiveView. If it fails, we send an error message. That’s it!

We now have a UI that looks like this:

We’re not really multi-node yet, but the moment we have a cluster and provide a list of real nodes, our :erpc.multicall will Just Work™ across the cluster and message our remote parent LiveView thanks to the distributed, location-transparent nature of Erlang’s process messaging.

For the LiveView to handle the timing messages, we only need to implement a few functions:

    def handle_info({_ref, {:loading, %Browser.Timing{} = timing}}, socket) do
  {:noreply, stream_insert(socket, :timings, timing)}
end

def handle_info({_ref, {:complete, %Browser.Timing{} = timing}}, socket) do
  {:noreply, stream_insert(socket, :timings, timing)}
end

def handle_info({_ref, {:error, {_, %Browser.Timing{} = timing}}}, socket) do
  {:noreply, stream_insert(socket, :timings, timing)}
end

  

Here we pattern match on the :loading, :complete, or :error tuples and call stream_insert to update the UI. The rest is some HTML markup and tailwind classes in our template to make it look pretty.

Going Multi-node

So what about going multi-node? Is it really that easy with the Erlang VM? Let’s take our dev prototype and deploy it on Fly.io:

fly launch

Once our app is deployed, we can scale out to any number of regions. For good geographic coverage, let’s spread things out across the planet:

fly scale count 8 --max-per-region 1 --region bom,fra,gru,hkg,nrt,ord,scl,syd

We use fly scale count to scale our app to 8 regions across the planet, ensuring we only have a single instance per region. We’ll start with Mumbai, Frankfurt, Sao Paulo, Hong Kong, Tokyo, Chicago, Santiago, and Sydney for nice world-wide coverage.

SSH'ing into any one of our Fly machines will show the Elixir nodes discovered themselves automatically:

fly ssh console --pty --command="/app/bin/wps remote"
Connecting to fdaa:0:36c9:a7b:98:e121:5775:2... complete

iex(worldpagespeed@fdaa:0:36c9:a7b:98:e121:5775:2)1> Node.list()
[:"worldpagespeed-01HW66QY8SHX7RF7S1A3519S4J@fdaa:0:36c9:a7b:e3:a239:a2f1:2",
 :"worldpagespeed-01HW66QY8SHX7RF7S1A3519S4J@fdaa:0:36c9:a7b:b4f1:eef5:616f:2",
 :"worldpagespeed-01HW66QY8SHX7RF7S1A3519S4J@fdaa:0:36c9:a7b:1d7:533d:e9ea:2",
 :"worldpagespeed-01HW66QY8SHX7RF7S1A3519S4J@fdaa:0:36c9:a7b:177:a43d:b898:2",
 :"worldpagespeed-01HW66QY8SHX7RF7S1A3519S4J@fdaa:0:36c9:a7b:fb:a2df:b6a1:2",
 :"worldpagespeed-01HW66QY8SHX7RF7S1A3519S4J@fdaa:0:36c9:a7b:f1:7795:8435:2"]
iex(worldpagespeed@fdaa:0:36c9:a7b:98:e121:5775:2)2> WPS.Members.list()
[
  {"bom",
   %{
     node: :"worldpagespeed@fdaa:0:36c9:a7b:177:a43d:b898:2",
     phx_ref: "F8j_UQe5Z2-nXQDh",
     machine_id: "9185750da6ddd8"
   }},
  {"ord",
   %{
     node: :"worldpagespeed@fdaa:0:36c9:a7b:98:e121:5775:2",
     phx_ref: "F8j_T5qu7dN9JQDh",
     machine_id: "5683566b095058"
   }},
   ...
]

We’re in business. Let’s try it out at https://worldpagespeed.fly.dev/.

It works! Our nodes clustered automatically and now when our LiveView hits the start_async call, :erlang.multicall will call each node to visit the provided URL. It’s simply message sending and receiving from there.

The code we wrote on our laptop works across the planet now – without changes.

This is great! But your webscale alarm bells might be going off. We’re putting our web UI, APIs, etc, in the hot path of a very resource intensive operation. Headless chrome eats hundreds of mb of memory and will churn CPU loading web pages. It will do this concurrently across requests to our app. How can we possibly scale this?

Surely AWS Lambda or Cloudflare Workers have some proprietary APIs to sell us for exactly this task? We’ll need to pay to configure an API gateway of course. We’ll also need to put the results somewhere like SQS, or use SNS to receive updates elsewhere. And pay for those too.

Or we can change two LOC and keep shipping.

Elastic scaling with FLAME

I often tell people that FLAME is so remarkable in how unremarkable it is when you use it in practice. What is effectively an amazing elastic scale primitive is turned into a boring decision.

“Do I want elastic scale here?”

If the answer is yes, you wrap your code in a FLAME.call and you carry on with life. That’s really the beginning and end of the decision process. You aren’t thinking about infrastructure or fractals of AWS glue to get the results back into your UI. You aren’t over-provisioning to sustain bursts, or adopting Kubernetes and microservices to service elastic load.

Let’s make this thing scale:

    def timed_nav(%Browser.Timing{} = timing, parent, ref) do
+ FLAME.call(WPS.BrowserRunner, fn ->
    timing = Browser.Timing.loading(timing)
    send(parent, {ref, {:loading, timing}})

    case Browser.time_navigation(timing, @browser_timeout) do
      {:ok, %Browser.Timing{} = timing} ->
        send(parent, {ref, {:complete, timing}})

      {:error, {reason, %Browser.Timing{} = timing}} ->
        send(parent, {ref, {:error, {reason, timing}}})

      {:error, reason} ->
        send(parent, {ref, {:error, {reason, Browser.Timing.error(timing)}}})
    end
+ end)
end

  

And we’re done.

We wrapped our timed_nav/3 function body in a FLAME.call/2. This will find or launch a fleeting instance of our application whose only job is to run this little slice of our app. Any state our function closes over is sent along to the ephemeral FLAME node. Any messaging we do inside via send(parent, ...) to the parent Just Works across the cluster because of course it does.

The fact that we sent our parent LiveView pid into a multicall, which sent it across the cluster to another node, who took the parent and sent it again to yet another node isn’t an issue . The message will make it back to the parent by design – it’s what the Erlang VM does.

Hot vs cold starts

We can start our FLAME pools hot or cold. For arguments sake, let’s say we have scale-to-zero behavior and an infrequently accessed app. What does our cold start time look like? We can update our app to show Starting browser on the local node when the user hits go, which will change to Loading page when the FLAME starts up with headless chrome. Here’s what that looks like:

Fly.io can start a fresh copy of our app in 3-5s, which includes headless chrome. We can also see how resubmitting the form catches the hot runners before they idle down from inactivity.

For this use case, cold starts aren’t an issue, but what if a three second cold start is too slow for your problem?

To avoid cold starts, we could configure our FLAME pool to always keep a single runner alive. Or we could configure our pool to warm up a number of runners at app start so our own cold deploys don’t cause users to hit a cold FLAME pool. Then those warmed up runners can idle down if no work is needed. The configuration to do that looks this:

    {FLAME.Pool,
 name: WPS.BrowserRunner,
 min: 1,
 max: 50,
 max_concurrency: 20,
 min_idle_shutdown_after: :timer.seconds(30),
 idle_shutdown_after: :timer.seconds(30)}

  

Here we start the app with a single warmed up BrowserRunner, and up to a maximum of 50 runners that each can service 20 concurrent web page visits. We also configure a 30s idle shutdown for the general pool operation, as well as the warmed up initial runner. The pool will grow and shrink elastically as load comes and goes.

Solving the problem vs removing the problem

Just imagine what this would have taken on your FaaS of choice.

To achieve elastic scale with “only pay for what you use!” pricing, you would stand up separate deployment pipelines of various lambdas, configure SNS or SQS queues, and write the glue back in your application.

You would also need to configure god knows what kind of availability zones and knobs to run your functions at the edge.

Or you could use FLAME and remove these problems entirely. With Elixir and FLAME you write the code and carry on with life. If you need to scale, you put it into a FLAME and start shipping your next features. FLAME machines on Fly.io launch by default in the same region as their parent, because of course they do.

It’s really as simple as that.

My ElixirConfEU talk covers the motivations for FLAME, as well as few other demos that are worth checking out.

You can also read more about FLAME in the original post, or check out the documentation for integration in your own applications.

Happy hacking!

–Chris

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

How Phoenix LiveView Form Auto-Recovery works

2024-04-09T00:00:00+00:00

We’re Fly.io and we transmute containers into VMs, running them on our hardware around the world. We have fast booting VM’s, so why not take advantage of them?

Phoenix LiveView is a server-side frontend framework and with this great power comes great responsibility on the developer. A very common source of frustration is around Forms, their assigns values and connection resets or deploys. In this post we’ll discuss how this all works and how it enables Form Auto-Recovery.

Let’s say we have this basic LiveView with a form and a query input:

    defmodule FormExamples.FormsLive do
  use FormExamplesWeb, :live_view

  def mount(_params, _session, socket) do
    {:ok,
      socket
      |> assign(form: to_form(%{"query" => nil}))
    }
  end

  def render(assigns) do
    ~H"""
    <.simple_form for={@form} phx-submit="save" id="form" class="mb-20">
      <.input field={@form[:query]} label="Search" />
    </.simple_form>
    """
  end

  def handle_event("save", %{"query" => q}, socket) do
    {:noreply,
      socket
      |> assign(form: to_form(%{"query" => nil}))
      |> put_flash(:info, "Form submitted with query: #{q}")}
  end
end

  

When we type into the text box and hit enter, our save callback is called and we reset the form. Before we move on, can you see the problem with this form?

On our Form submit we are assigning "query" => nil but "hello" remains in our text box. What’s going on here?

Spoiler: We did not change the assign value while we were typing the query, in LiveView’s view of the world the form‘s query value is nil, and it never changed. These implications will be discussed further in the article.

The Problem of State

LiveView and, any good frontend framework, help us organize our front end code around encapsulating state within a component. Meaning the only place we need to think about the state of a view is in our assigns, there will be no other references mutating our values outside of the component.

Unfortunately in all frameworks this a lie, or a helpful fiction. All frontend frameworks actually have to deal with three(!) separate local states and keep them in sync, for example here are the 3 states for LiveView:

Server, where we keep track of assigns, session and the WebSocket related state.
Browser HTML DOM state.
Browser Internal DOM state.

In react the first state would be the Component/state/props, they still need to maintain the HTML and the internal DOM State. This is why they have so many functions around use* now to encapsulate the many different ways you might want to manage state and the dom.

In LiveView flow of state is as follows:

Developer changes assigns.
LiveView calculates the minimum HTML Diff changes and sends it down the WebSocket.
LiveView.js sends the changes to the virtual DOM library morphdom which alters the HTML DOM.
User changes inputs, which only changes the Browser Internal DOM State.

Knowing this maybe we can guess what went wrong with our form, and it happens right at point 1, the assigns value for the query value never changed. No diff was calculated, the browser HTML never changed so the HTML Input value never changed, all the while the Browsers Internal DOM State value did change.

Here is how we can change our example to reset the value on submit:

    defmodule FormExamples.FormsLive do
  use FormExamplesWeb, :live_view

  def mount(_params, _session, socket) do
    {:ok,
      socket
      |> assign(form: to_form(%{"query" => nil}))
    }
  end

  def render(assigns) do
    ~H"""
    <.simple_form for={@form} phx-submit="save" phx-change="validate" id="form" class="mb-20">
      <.input field={@form[:query]} label="Search" />
    </.simple_form>
    """
  end

  def handle_event("validate", %{"query" => q}, socket) do
    {:noreply, assign(socket, form: to_form(%{"query" => q}))}
  end

  def handle_event("save", %{"query" => q}, socket) do
    {:noreply,
      socket
      |> assign(form: to_form(%{"query" => nil}))
      |> put_flash(:info, "Form submitted with query: #{q}")}
  end
end

  

Here we added a phx-change to our form, with handle_event function that changes the form and query assigns whenever the form changes. Now when we hit submit our input would reset to nil because every single keypress we informed LiveView of this change! We can follow the flow of state here:

User changes the text box, firing a phx-change event.
Handle event assign changes the LiveView state.
LiveView calculate s a diff that is essentially set input.value to assigns.query
LiveView.js and morphdom sets the HTML DOM value of our input.
Which is the same as the current HTML Internal Dom value so no change happens that the user can see.

Enter A Server Restart

Now this is where the Auto-Recovery Magic happens when we deploy or the process restarts:

On re-connect a phx-change event is fired with the current DOM for every Form that has the same id as a reconnect mounts html.
Handle event assign changes the LiveView internal state with the validate params.
LiveView calculate s a diff that is essentially set input.value to assigns.query.
LiveView.js and morphdom sets the HTML DOM value of our text box.
Which is the same as the current HTML Internal Dom value so no change happens that the user can see.

The only difference here a little bit of Magic that LiveView does when it knows it’s reconnecting to an existing DOM State. If the current DOM has a Form with the same id, instead of replacing it, it fires the change event which should alter the LiveView’s current assigns with the previous state! And there you have it—fully recoverable forms with no server-side state at all! This works assuming the Form’s id remains unchanged.

And if you don’t believe me I pushed a live example here with code here.

Takeways

Always give each Form a unique and consistent id.
Always have a phx-change event for Forms that updates the server side assign.

By understanding its inner workings of form handling and auto-recovery mechanisms, we can unlock the full potential of LiveView to create seamless user experiences. I encourage you to check on your own forms and inputs and make sure they have ids and phx-change events wired up to give your users a better user experience!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Easy at-home AI with Bumblebee and Fly GPUs

2024-04-02T00:00:00+00:00

We’re Fly.io and we transmute containers into VMs, running them on our hardware around the world. We have fast booting VM’s and GPUs; so why not take advantage of them?

A big barrier to getting started with local AI development is access to hardware. And by “local”, we mean having direct access to a GPU and not going through AI-as-a-Service. Some of us are lucky enough to have a beefy Nvidia GPU, if so, good for you. For the rest of us, there are other ways.

Llama.cpp - LLM inference in C/C++ which can run reduced models on CPUs.
ChatGPT - represents the category of AI-as-a-Service. We can’t run our own models.
Replicate - represents platforms that let you run open-source AI models.

But Elixir has the ability to run and host some pretty interesting open source models directly through Bumblebee. For the big ones, you need access to a GPU, or you have to be really, really, …really… patient.

For those of us who don’t have the hardware locally, we can run a GPU on Fly.io while editing the app on our machine.

Sounds crazy? It’s actually really cool! And when it’s up and running, it genuinely feels like you have a fast GPU plugged directly into your laptop.

What exactly are we talking about?

In a nutshell:

Take a ready-to-deploy minimal Elixir application and deploy it on Fly.io.
Get a GPU and volume for persistent storage to cache the multi-gig AI models on disk.
Load up our local Elixir application, cluster it to the app on Fly.io through the VPN, using Elixir’s Nx library
Start developing AI features in a local app where the AI part runs on a remote GPU. Nx makes the distributed calls transparent to us when we’re clustered.

We can visualize it like this:

Let’s get started!

An Elixir app to host our model

For Bumblebee to really shine, it needs a GPU attached. A major goal here is to keep the app on the server as simple and small as possible so we don’t need to re-deploy it.

This means all the active development of the app stays on our local machine! 🎉

The thin app on the server is just a harness for hosting our model with Bumblebee. It contains no business logic, has no UI, and basically does nothing but host the model and provide an Nx Serving we can talk to.

Get and deploy the app

Getting the ready-to-deploy harness application is as simple as:

git clone git@github.com:fly-apps/bumblebee-model-harness.git
cd bumblebee-model-harness

At this point, you can open the included fly.toml file and change the name of the app to be unique and whatever you want. Just hold on to that new name! We’ll need it shortly.

Then, continue with:

fly launch

...
? Would you like to copy its configuration to the new app? Yes

This builds the Dockerfile image, deploys it, and starts the selected serving. For me, the process of starting the serving for a new Llama 2 model took about 4 minutes to download and start.

You can watch the logs to see when it’s ready:

fly logs

The following log lines are part of a healthy startup. Note: other lines were excluded.

2024-03-27T02:36:12Z app[3d8d79d5b24068] ord [info]02:36:12.491 [info] Elixir has cuda GPU access! Starting serving Llama2ChatModel.
2024-03-27T02:40:12Z app[3d8d79d5b24068] ord [info]02:40:12.245 [info] Serving Llama2ChatModel started
2024-03-27T02:40:37Z app[3d8d79d5b24068] ord [info]02:40:37.157 [warning] Nx.Serving.start_link  - {:ok, #PID<0.2164.0>}

At this point the server is ready and we need a client to use it!

A client Elixir application to use it

Let’s create a new Phoenix application for this. No database is needed.

mix phx.new local_ai --no-ecto
cd local_ai/

I’m using direnv to manage my ENV settings on a per-project basis. Dotenv does this just as well.

We’ll create an .envrc file with the following contents. It tells our local application which of our deployed Fly.io applications to cluster with. This is where your chosen app name comes in!

    export CLUSTER_APP_NAME=my-harness-app-name

  

Next, add {:nx, "~> 0.5"} to mix.exs and run mix deps.get

Start and cluster the local app:

./cluster_with_remote

The linked guide walks you through setting up a VPN connection and creating your own local ./cluster_with_remote script file.

The following is logged when successfully starting the local Elixir project and clustering it with the harness app:

Attempting to connect to my-harness-app-name-01HSYVSNTG4XT8TPPGXFP6RJ66@fdaa:2:f664:a7b:215:875f:cb5d:2
Node Connected?: true
Connected Nodes: [:"my-harness-app-name-01HSYVSNTG4XT8TPPGXFP6RJ66@fdaa:2:f664:a7b:215:875f:cb5d:2"]

With the local application running and clustered to the server, it’s time to start coding AI features!

Running local code on the GPU

As a simple getting-started example, copy and paste this into the IEx terminal.

    stream = Nx.Serving.batched_run(Llama2ChatModel, "Say hello.")
Enum.each(stream, fn
  {:done, token_data} ->
    IO.puts("")
    IO.puts("DONE!")
  data ->
    IO.write(data)
end)

  

It talks to the Llama2ChatModel, which is the name of the serving we enabled in the harness application. We ask it generate text based on our "Say hello." initial prompt.

The serving returns a stream which we enumerate over and write the data out to the console as it’s received. It writes out something like the following in the terminal:

Through the grapevine.
Say hello to the person next to you.
DONE!

That’s all it takes to start working with a self-hosted LLM!

The locally running application is the one we actively develop. It has all our business logic, UI, tests, etc. When the application needs to run something on the LLM with a GPU, it transparently happens for us and the response is streamed back to our application!

We really do get to keep our normal local-first development workflow, but now with powerful GPU access!

What’s can I do next?

Check out the Bumblebee docs for examples and lots of models to start playing with.

If working with LLMs is your interest, check out the Elixir LangChain library and the ChatBumblebee model to add chat message structures. It makes Bumblebee-based conversations easy.

Here’s a quick demo of a project running locally on my dev machine but executing the chat completion through the harness app running on a Fly.io server with GPU.

Did you notice how rapidly the text is generated? That would NEVER be possible on my local machine.

Setting some Bumblebee expectations

Functions are required for getting the most out of an LLM. That’s what enables the LLM to directly interact with your application.

At the time of this writing, function support is not yet reliably possible with Bumblebee. Bumblebee needs the ability to constrain the generated text to be only valid JSON, which at the time of this writing, it can’t.

There are some ways around it, but that’s beyond the scope of this post. It might be fun to explore this further. Hmm. 🤔

Develop as you would normally

This is where you win. We keep the same basic workflow that we are already comfortable and productive with, except with now we’re doing it with a remote Fly.io GPU too!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. We’ve got a GPU ready for you to take on a test drive. Buckle up!

Deploy a Phoenix app today! →

Tips and Tricks

Working with GPUs, Nx, Bumblebee, HuggingFace, and CUDA cores is still new to most of us. This section covers some tips that were learned personally by hitting one wall after another. Take head and keep your head unbruised!

Note

All the tips related to the server are already in the harness application. They are included here as an explanation of the change for making them on other projects.

Server: Dockerfile needs all the Nvidia dependencies

See the Fly GPU quickstart guide for Dockerfile examples. The harness GitHub repo includes a working Dockerfile as well.

Server: Turn off Bumblebee’s console progress logging

When using Bumblebee locally, it’s handy to have the progress reported in the console for large model file downloads. However, in a server environment, this breaks the IO stream for the application logs and crashes the server. Additionally, no one is watching the console on the server anyway.

To address this issue, we add the following to our config/prod.exs

    config :bumblebee, progress_bar_enabled: false

  

Server: Delay starting the Nx.Serving

In the harness application, I created a Harness.DelayedServing GenServer that spawns a separate process to start the start the Nx.Serving.

The purpose of this GenServer is to start the Nx.Serving for the desired module. Large models can take several minutes to download and process before they are available to the application. This GenServer, or something like it, can be added to the Application supervision tree. It detects and logs if Elixir has CUDA access to the GPU. If support is available, it starts the serving asynchronously and makes it available to the application.

That is the sole purpose for this module. Without this delayed approach, the extended start-up times can result in an application being found “unhealthy”, and killed before ever becoming active.

An alternative approach is to transfer the needed model file to the server in some other way.

Server: Enable clustering on the server

The Phoenix library dns_cluster is an easy option to enable clustering on the server. It’s built-in to new Phoenix applications

Client: Make it easy to reconnect to the server

In order to take advantage of a GPU on Fly.io, we need to be clustered with an Elixir application running there.

The guide Easy Clustering from Home to Fly.io documents how to get that setup. To automate the process of connecting to the server, use the shared bash script to start our local application and cluster it with the server.

Phoenix Dev Blog – Server logs in the browser console

2024-03-22T00:00:00+00:00

We’re Fly.io and we transmute containers into VMs, running them on our hardware around the world. We have fast booting VM’s and GPUs; so why not take advantage of them?

There are some features that are such quality of life improvements I’m shocked we didn’t ship them sooner. The latest phoenix_live_reload which streams servers logs down to the client is one such example.

The phoenix_live_reload project is about as old as Phoenix itself. It has been around for almost 10 years as a humble dev dependency that refreshes your browser or updates your stylesheets whenever your files change on disk. We dog-food Phoenix’s built-in bidirectional cli/server communication layer to make this happen.

When your web framework does realtime sockets out-of-the-box as a core feature, you can do a lot for free. So why stop at merely refreshing the browser?

The Problem

Anyone building web applications knows this grind – you’re plugging away on a feature and clicking around the browser. Something breaks. Maybe the UI shows some kind of error state. Maybe nothing happens when something should have happened. You check the browser’s web console for errors. Maybe our js threw an error? Nope? Okay but it didn’t work like we expected, so did the server send what we expected? Let’s check the browsers network tab. Hmm looks good. Okay let’s check the server logs back in whatever terminal we are using. Time to scroll through a thousand lines of scroll-back and look for our logged errors.

This is a problem whether you’re working on a SPA, a LiveView project, or those dusty jQuery $.ajax() calls on some legacy workhorse. The UI is necessarily coupled to the server, and either can fail or have bugs.

It’s time to cut down on the window flailing debug experience.

The Solution

It’s obvious. Let’s collocate the server logs with the client logs! The first place you’re going to look while using the UI is the browser’s web console. Your UI framework logs and UI errors are already there and built into your workflow. Interlace your server logs there, and it becomes a one-stop-shop of useful info.

The implementation is boring. We already have Phoenix channels for trivial bidirectional communication, and we already have a live reload channel running in dev for all phoenix applications. We add a custom Erlang log handler on the server and use Elixir’s Registry to broadcast log events to channel processes. Those channels push the logs down to the client and effectively console.log with some colors based on log level. The result is dev bliss:

Erlang makes it easy to hook into the VM logger. The entire meat of the live reload log shipping mechanism is less than 30 LOC:

    defmodule Phoenix.LiveReloader.WebConsoleLogger do
  @registry Phoenix.LiveReloader.WebConsoleLoggerRegistry

  def attach_logger do
    :ok =
      :logger.add_handler(__MODULE__, __MODULE__, %{
        formatter: Logger.default_formatter(colors: [enabled: false])
      })
  end

  def child_spec(_args) do
    Registry.child_spec(name: @registry, keys: :duplicate)
  end

  def subscribe(prefix) do
    {:ok, _} = Registry.register(@registry, :all, prefix)
    :ok
  end

  # Erlang/OTP log handler
  def log(%{meta: meta, level: level} = event, config) do
    %{formatter: {formatter_mod, formatter_config}} = config
    msg = IO.iodata_to_binary(formatter_mod.format(event, formatter_config))

    Registry.dispatch(@registry, :all, fn entries ->
      event = %{level: level, msg: msg, file: meta[:file], line: meta[:line]}
      for {pid, prefix} <- entries, do: send(pid, {prefix, event})
    end)
  end
end

  

The attach_logger function uses Erlang’s :logger.add_handler/3 to attach our logger. This will call the log/2 function every time a log occurs. In our log function, we use Elixir’s Registry.dispatch/3 to act as a local pubsub and notify the live reload channels whenever a new message is logged.

This has been released with {:phoenix_live_reload, "~> 1.5"}, and you can try it out by enabling the web_console_logger in your config/dev.exs live reload config:

    config :my_app, MyAppWeb.Endpoint,
  live_reload: [
+   web_console_logger: true,
    patterns: [
      ~r"priv/static/.*(js|css|png|jpeg|jpg|gif|svg)$",
      ~r"priv/gettext/.*(po)$",
      ~r"lib/my_app_web/(controllers|live|components)/.*(ex|heex)$"
    ]
  ]

  

Then in your assets/js/app.js enable the server logs when live reload is attached:

    window.addEventListener("phx:live_reload:attached", ({detail: reloader}) => {
  // Enable server log streaming to client.
  // Disable with reloader.disableServerLogs()
  reloader.enableServerLogs()
  window.liveReloader = reloader
})

  

That’s it! Happy hacking!

Clustering Elixir From Laptop to Cloud

2024-03-21T00:00:00+00:00

We’re Fly.io and we transmute containers into VMs, running them on our hardware around the world. We have fast booting VM’s and GPUs; so why not take advantage of them?

TL;DR: Check out the Elixir Fly.io guide Easy Clustering from Home to Fly.io for the steps on exactly how to cluster the Elixir application running on your laptop with one running at Fly.io. Grab the bash script that automates a chunk of it too. Here we talking about what this means and why we might want to do it.

What if you could cluster the Elixir app running on your laptop with the one you deployed on Fly.io? Would you want to? What would you do if you could? Because, you can, you know.

What do we mean by “clustering”?

If you are newer to Elixir, clustering isn’t creating a web API and making calls to another server. The concept can feel foreign as not many languages have this built-in feature.

Elixir actually inherits this ability from Erlang. Erlang is a concurrent, functional programming language designed for building scalable and fault-tolerant systems. It was developed by Ericsson for use in telephony applications but has since been adopted by numerous industries for applications requiring high availability and concurrency.

“Clustering” here refers to the practice of connecting multiple Elixir/Erlang runtime systems (nodes) over a network into a single distributed system. These clustering capabilities are a core part of the language and the OTP (Open Telecom Platform) framework, which provides a set of libraries and design principles for building robust systems.

In practical terms, this means we can have two Elixir applications running on two separate machines and connect them in a seamless way. Once connected, we can execute arbitrary functions on another machine and even send messages to processes in a way where the application code doesn’t need to know about the other machine at all!

Why cluster the app on my laptop with one on the server?

Clustering is already widely used for sharing PubSub messages across machines. It can be used for a lot of things actually. It lets our applications assign work to another node that is better suited to the task. We don’t have to assign work, perhaps we want to monitor and interact with a running application on another machine. We can do all that and more.

Let’s explore a few reasons we might want to do this:

Developing AI/ML applications and leveraging more powerful hardware.
Developing and testing distributed applications.
It’s cool.

1. Developing AI/ML applications

Right now, “AI” is all the rage. Got an AI startup idea? VC’s may throw money at you just for saying so. But developing an AI application may be more intensive and, depending on what you have in mind to build, may require more specialized hardware than what comes packed in your laptop.

We can offload the AI/ML work to a machine with a GPU, but develop our application normally on our laptop.

More to share on that in the near future!

2. Developing distributed applications

Building a globally distributed application can be challenging to model locally. With Fly.io Regions, we can deploy our cluster-aware application where it makes sense. Then, our local application joins the globally cluster, giving us a close-up view of how the application behaves in a truly globally distributed environment.

3. It’s cool

Okay, “it’s cool” isn’t a good business reason, but come on… this is cool! Sometimes we need to play with things for sake of play. When we’re playing, we’re more likely to ask ourselves a question like “Huh, I wonder if…”, which can lead us to interesting personal breakthroughs.

How do we do it?

A Fly.io Elixir guide was added detailing how to do this. In the Easy Clustering from Home to Fly.io docs, there’s a bash script that automates much of the process, making it easy to start a local Elixir application and cluster it with one running on Fly.io.

Here’s how it works:

Fly.io’s WireGuard VPN network connects the laptop to the Fly.io network
Elixir application should be setup for clustering
Run the same version of Elixir and Erlang on both ends
Configure and run the script!

Steps 1-3, are a one-time setup. Once that’s worked out, it’s just:

Open the VPN connection
Run the script

Conclusion

Connecting the Elixir application running on our laptop to an Elixir application running on the server isn’t difficult. We can cluster the same or different applications together.

There are even practical reasons to do it. We can offload AI/ML tasks to a machine with a GPU and develop our applications as per usual. We can develop and debug truly globally distributed applications right from our laptop. And, it’s freaking cool.

Check out the documentation to walk through the process.

What are you going to play with?

What if S3 could be a fast, globally synced, Key Value Database? That's Tigris

2024-02-20T00:00:00+00:00

We’re Fly.io and we transmute containers into VMs, running them on our hardware around the world. We have fast booting VM’s, so why not take advantage of them?

That said… building applications that span the globe is a hard problem. Heck, syncing data between two machines is not trivial. It might require writing to a “primary” and reading from a “secondary”. Or using a CRDT to “sync” state, combined with pub-sub to communicate changes.

There’s a wide selection of databases and libraries that claim to solve this class of problems. With the caveat that we need to operate this system under our application. Or, pay someone else to operate it for us. While Elixir/Phoenix has tools to make this easier, it’s no free lunch. You need to consider the many failure modes that could corrupt your data.

One solution: give up and choose a central actor to store data. Most apps today already use S3-style object storage this way. Instead of storing files in a database, we accept that we have a globally-accessible location (bucket) that stores values (file data) identified by keys (file names).

So, why don’t we treat S3 as a simple key-value store? We have cheap, strongly-consistent, bottomless storage, with wide language support.

This comes down to the speed of light. S3 makes it trivial to store as much data as you want in a single region, in a single bucket. Which is what almost every user of S3 does, this leads to a “US-EAST Privilege” for users. Closer you are to US-EAST the better your experience with the web will be, the further away the slower every website feels. This can be ameliorated with CDN but now we’ve added more cost and services to our stack.

Enter Tigris

Tigris is an S3 Compatible file storage built on Fly.io, for Fly.io users, with interesting properties:

Objects are initially stored in the region closest to the uploader say Chicago, where they are most likely to be served.
If that object is requested from a Singapore, Tigris will transparently move, then cache the object there. Like a CDN you don’t have to configure.
Every file under 128kb is cached globally.

How they do that is beyond the scope of this post but we recommend reading through their architecture documentation, its very good!

I am very interested in that last bullet point, if every object under 128kb is cached and synced globally that means we have a globally spanning Key Value store that should be crazy fast to read and write to! So let’s build a simple KV Store against Tigris.

Experiment

Building off my previous post where I broke down and built our own AWS client, I’ll be using a similar extension to Req I built as shown in this gist. Our entire api will be something like this:

    KV.get("/path/to/key") # get key contents
KV.put("/path/to/key", value) # update/create
KV.put_transaction("/path/to/key", value) # update/create with global lock on key
KV.delete("/path/to/key") # delete
KV.get("/path/to/") # list keys

  

I will spare you the details here as this is literally just reading and writing to a file and encoding the value using :erlang.term_to_binary.

Keen observers might realize we have one glaring issue and that’s data races. If we have a globally spanning cluster and two regions write to the same value at the same time, we’ll have a conflict.

To solve this, there is one function in our API that’s not like the others with put_transaction. Incredibly the BEAM comes with a function that helps us out here called :global.trans/3. It will create a “global lock on an id” and execute your callback ONLY on a single node. Our put_transaction code literally looks like:

    def put_transaction(key, value) do
  :global.trans(key, fn ->
    put(key, value)
  end, 0)
end

  

Now we have globally distributed transactions on our KV store. Normally the trans function will retry forever until it gets a global lock, in our code we tell it to not retry we’ll fail here with :aborted and let our user’s figure out what to do.

And that is that! We’re using the key based “File System” of Tigris as a KV store and so long as our values are under 128kb we’re syncing as fast as the internet allows!

Setup Tigris

Assuming you are using Fly.io already, adding Tigris to an existing app is as easy as

$ fly storage create
? Choose a name, use the default, or leave blank to generate one: demo-bucket
Your  project (demo-bucket) is ready. See details and next steps with:

Setting the following secrets on app:
AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
BUCKET_NAME
AWS_ENDPOINT_URL_S3

Secrets are staged for the first deployment

And that is it we have full access to the Tigris API and the common configuration variables already set. As simple as that we have a functional KV store thats durable, globally cached and for cheaper than AWS.

Brainstorming

Let’s brainstorm what else we could achieve with this, noting this is a brainstorming session no bad ideas, and not production promises.

Durable Processess

Since the read/write latency is low we can simply write our process state out to Tigris every few seconds and or on terminate. The next time the process or LiveView is loaded we have our last known good state.

Tables and Indexes Oh My!

The API is just a file system, we could use key paths as a sort of table spaces Req.get("s3://kv-bucket/users/") or create indexes of data we need to query often Req.get("s3://kv-bucket/users/idaho") keeping those indexes up to date via a Worker is an exercise for the reader.

Single Tenancy

Maybe your user’s need exclusive single tenancy, you could put_transaction a SQLite DB into a folder with a .lock key pointing to the fly machine-id that owns it. When the first user opens up the website, download db and put_transaction the .lock file. When the next user opens up the page in a new region, check the lock and redirect them to the machine that owns the DB with fly-replay. Syncing everything periodically or when the last user closes the page you’ve got a globally distributed SQLite. Obviously this would fall over if you have too much concurrency, racing to read/write the lock file, but for smaller users this could work well!

Append-only tables.

Every machine can create new files with their data in a common directory. Another machine could query that list of files and collate them into a common thread.

Wrap-up

Tigris has given a rare gift, a new way to look at an old tool, S3. Globally distributed, close to our users and fast global sync for files smaller than 128kb. Not only can we simply replace S3 in our existing stack’s we can use it in new and fresh ways!

I can’t wait to see what you build with Tigris!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Not every Dependency is worth it.

2024-01-30T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Elixir has been quietly building out a truly impressive list of packages, and while a deep resource of packages that solve common or complex problems is good, for some people it’s not enough and want more. Which is to be expected, the Elixir community is not the same size as Python or JavaScript, we don’t have client libraries for every single service or startup or library. While a deep set of client libraries is handy for quick iterations, it can gradually turn into a liability. With a little thinking and time at the keyboard we can often come to a solution that fits our problem neatly with something we can fully understand.

As Gary Bernhardt recently posted on twitter

Over the last ~5 years or so I’ve worked hard to:

- Remind myself that “this problem isn’t my fault, but it’s my responsibility” when appropriate.
- When a problem is my responsibility, aggressively make it impossible for the problem to recur.

Great combo; recommended.
— Gary Bernhardt (@garybernhardt) January 20, 2024

it is always a good reminder, that our dependencies are our problem.

In this post today, we’ll walk through how I evaluate dependencies and then discuss how I go about replacing them in my own projects.“

S3

At some point every application needs a place to put files either for long term backup or public consumption, and for the time being S3 is the place to do it. AWS API’s are fairly notorious for being a little obtuse to work with so the first instinct is to grab a package, time to search Hex.pm.

29 total packages and while this is specific to s3, if we search aws instead we’ll find 150 total packages and the currently best supported option of aws. Which is an Erlang project to autogenerate the AWS Library using aws-codegen into modules and functions we can call.

This is a red flag for me as in my experience I find the "code-gen” style libraries a little obtuse to work with, and usually poorly documented. Let’s give the benefit of the doubt though and give it a swing!

A good practice before adding a package is to dig into the dependency tree to make sure we know what we’re getting into.

aws_signature
finch (http library I chose)
- castore
- mime
- mint
  - castore
  - hpax
- nimble_options
- nimble_pool
- telemetry
hackney (optional not used)
jason
- decimal

Most of these packages are fairly familiar and eventually are included is nearly every application as they grow. One exception is aws_signature, let’s take a peek into that quick. It has no dependencies which is great and the description is:

Request signature implementation for authorizing AWS API calls.

Which is great because that step is often one of the more confusing and complex parts of using the AWS API. Taking a quick pass over the code it looks like something I would write to sign some AWS code, and is well tested/documented. Great!

This has taken all of a couple minutes and we’ve learned a few interesting things:

aws uses code-gen for code and docs. Which is a great way for a consistent and up to date API, it’s not great from a UX Perspective.
The aws package appears to allow for configuration of HTTP Client’s which is a good sign, we’ll get in to why later.
The existence of the aws_signature library, made by the authors, is noteworthy. This is a handy library for when working with AWS, and can be an error prone process if done manually.

Luckily for us the authors of the aws library have S3 examples in the README! I’ll reproduce the code from their example below we can talk about it:

    iex> client = AWS.Client.create("your-access-key-id", "your-secret-access-key", "us-east-1") |> AWS.Client.put_http_client({AWS.HTTPClient.Finch, []})
iex> file = File.read!("./tmp/your-file.txt")
iex> md5 = :crypto.hash(:md5, file) |> Base.encode64()
iex> AWS.S3.put_object(client, "your-bucket-name", "foo/your-file-on-s3.txt",
  %{"Body" => file, "ContentMD5" => md5})

  

Here we see them creating the AWS.Client, setting Finch to the http client, reading the file into memory, hashing it as an optional step and putting it up to their bucket with filename.

This is actually extremely good, in three lines we’re able to upload file, with minimal fanfare. Frankly, I was expecting way more configuration steps and navigating poorly written docs to get this far. If you decide to stop here and use aws for your S3/AWS needs, I believe you’d be making a reasonable and sound choice.

Let’s dig into the pros and cons of this library as I see it.

What has `aws` done well?

Avoids HTTP Client Bloat

One issue that arises as applications grow is what I like to call ‘http client bloat’. Every dependency that wraps a 3rd party service uses their own HTTP Client, with its own dependencies, connection pools, processes, telemetry and error handling. This makes configuration and observability very difficult.

aws is explicit about requiring you to choose your own client, giving you two sensible defaults and an escape hatch to include you own.

Bare bones API with few Custom Structs/API’s

Library authors of client dependencies love to add “idiomatic” data structures to their client’s that make their project “feel” more native. Often what happens is it obscures the underlying API and slowly drifts as the API changes and the author loses interest in maintaining every single little change. I won’t call anyone out specifically but there are several highly used Elixir client packages that commit this crime.

The authors to aws have done a good job here, the struct’s they created are generic to all aws services and the generated code makes use of simple map‘s for responses. This let’s you navigate the AWS docs and see the same key names you’d expect in the request and response. Without converting them to someone else’s hand made Structs.

What is so bad about `code-gen`?

From a package maintainer perspective it is fantastic. We can put out a fully functional client library, with functions and arguments and docs all validated at compile time, and with minimal effort.

From a user perspective, we can identify two problems: the generated code and documentation are often subpar in fitting the language, and the library suddenly includes the entire AWS ecosystem

To illustrate the issue of the generated code let’s look at a screenshot of the AWS hexdocs page:

While nothing is empirically bad here, it’s worth noting that, although I needed an S3 package, the entire AWS ecosystem is now part of my dependency tree! Every single API is here and even if I don’t use them I now download and compile them.

Finally the module docs in Elixir projects typically include examples, explanation, gotcha’s and advice for using the module, we don’t get that here. Outside of the README examples it’s mostly up to you to figure it out. One nice bit here is when you click into the function you will get the whole REST API Documentation from AWS for that function which is nice, not always helpful, but better than clicking around the AWS website!

Let’s take ownership

Thing’s happen and in the real world all project dependencies slowly become liabilities. Security issues creep up, maintainers burn out, bugs get introduced and general lack of budget towards maintaining all code catch up to us.

Let’s imagine we’ve had enough, and while this package is great, we want to take ownership of our very small usage of S3 into our own hands. The surface area is small and we know how to use an HTTP Library, Like Gary said

“Its not our fault, but it is our problem”

and if its our problem, lets own it.

Prior Art

Whenever I go down this path I like to take a minute and look for some prior art. And these days the best place to look can end up being the LiveBook project. I know they have S3 Integration and sure enough they have their own hand rolled client! Coming in at a paltry 360 lines of Elixir plus a bunch of code we don’t care about handling XML response parsing.

Zooming in the LiveBook team also use the aws_signature library and have a configurable http client. Further if we look at just the code we care about its under 70 total lines that do what we want! Let’s take a look at the core if the request function:

    url = build_url(file_system, path, query)
headers = sign_headers(file_system, method, url, headers, body)
body = body && {"application/octet-stream", body}

result = request(method, url, headers: headers, body: body, timeout: timeout)

When building our own application this will the basis for our own client!

Step one build the url which included bucket name + key and query string.
Create the signed headers, which the aws_signature library does most of the heavy lifting for us there.
And finally make the request!

Implementation

You won’t be surprised to know that I’m a big fan of the Elixir HTTP client Library Req. Req is a high level abstraction over top of Finch the slightly more barebones HTTP Client. It describes it’s self as the “batteries included” HTTP Client, and its what I’ll be using here.

    def put_object(credentials, bucket, file_name, mime, file) do
  now = NaiveDateTime.utc_now() |> NaiveDateTime.to_erl()
  host = "https://#{bucket}.s3.amazonaws.com"
  url = "#{host}/#{file_name}"
  headers = [{"Host", host}, {"Content-Type", mime}]
  headers =
    :aws_signature.sign_v4(
      credentials.access_key_id,
      credentials.secret_access_key,
      credentials.region,
      "s3",
      now,
      "put",
      url,
      headers,
      file,
      uri_encode_path: false,
      session_token: Map.get(credentials, :token, nil)
    )

  Req.put(url, headers: headers, body: file)
end

  

And boom! We have a fully functional and “minimal” AWS.S3.put_object function! There are numerous ways we could improve this and make it more generic, support more options or functions but this is all we needed! If we need more, we have prior art to with LiveBook to build off of, if we don’t need more we’re done!

Wrap up

While this may feel like a contrived example I assure you it is not. Every HTTP-based API client library has its own idiosyncrasies, but once you get past the basics, you can implement any service API and gain full control with minimal fuss.

What dependencies have you included to only end up using one function? Could you re-implement that function yourself? Assuming the license is open could you maybe vendor JUST the functions you need?

Once we start down this path we will be asking ourselves these questions constantly. We are programmers and we can solve problems with code! We don’t simply need to “plumb” together several libraries, and we might benefit from it!

Look into your own codebases, logs, and error handling and find troubled dependencies that you can learn more about and possibly take ownership for!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Making Phoenix LiveView Sing!

2024-01-24T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Let’s be real. You’re probably never going to create a 3D game in Phoenix LiveView. It’s not the best tool for that job, at least yet. However, we can still create interesting, compelling, and responsive games in LiveView!

What’s a fundamental piece of games? Sound. Yes, the simple “nom-nom” sound effects that accompanied the early Pac-Man game were an important part of the experience.

What? You don’t care about creating a web-based LiveView game? Not everyone does or should! But adding sound effects to a web application can communicate additional information. In fact, sound effects can convey meaning on their own. Have you noticed that most games even use sound effects in their settings pages? Why? Because it’s part of creating an immersive experience. It connects the user to the interface in an additional dimension.

Before we dig in, let’s see a demo of what we’re talking about.

Click here for the demo source code.

Notice how when the count incrementing button can’t go higher, the sound emitted from a click changes, communicating more about the state and the effect of the user’s action.

Additionally, the button triggering a delayed sound demonstrates how the server can push an event from the LiveView to the browser, triggering a sound event based on the server’s state.

Now, let’s make some noise!

How do we add sound?

An obvious place to start is with the HTMLAudioElement which “provides access to the properties of <audio> elements, as well as methods to manipulate them.”

This works fine for embedding an audio file like a podcast. The browser renders play/pause controls along with playback speed settings. Nice!

But this isn’t what we want for our simple sound effects.

Turns out there’s another Web Audio API that’s better suited for our purposes. From the W3C spec’s introduction:

The introduction of the audio element in HTML5 is very important, allowing for basic streaming audio playback. But, it is not powerful enough to handle more complex audio applications. For sophisticated web-based games or interactive applications, another solution is required.

While this gives us more granular control, it’s at the cost of added complexity. We are responsible for setting up and managing an AudioContext that other sounds are associated with. That ends up being a bit of a headache.

But we only want to play a simple sound effect! Isn’t there an easier way?

Mobile audio munging

Our usage of audio sound effects is more aligned with web based games. Mozilla has some great resources regarding Audio for Web Games. One of the first things they call out is the challenges with mobile devices. Here’s what they say:

By far the most difficult platforms to provide web audio support for are mobile platforms. Unfortunately these are also the platforms that people often use to play games. There are a couple of differences between desktop and mobile browsers that may have caused browser vendors to make choices that can make web audio difficult for game developers to work with.

Gah! It’s just getting worse! How can we solve this?

JS libraries to the rescue!

Fortunately for us, there is an easier way! There are a number of JS audio libraries available that help address all this complexity.

Which is the “right” library depends on the use case. If we were building a DJ-style mixer, we’d want a different sort of audio control and manipulation. The library we’ll use for our needs is Howler.js. The project describes itself this way:

howler.js is an audio library for the modern web. It defaults to Web Audio API and falls back to HTML5 Audio. This makes working with audio in JavaScript easy and reliable across all platforms.

Phew! Sounds like it’ll do the trick!

We need sounds to play!

Before we go further, we need some sound effects to play. There are many sources of sound effects online, or we can even create them ourselves. For our simple needs, we’ll turn to the sound effects section of OpenGameArt.org.

After choosing some sounds, trimming them down tighter, and exporting them as MP3 files, we’re ready to move on to the next step.

Hook-ing up our audio

Phoenix LiveView supports JavaScript interoperability to let us connect the JS library (Howler.js), to our LiveView page.

This next part links to the demo project source code for the details.

Add the audio files to the directory priv/static/audio.
Add audio to the list of static paths
Add assign_sounds to the LiveView’s mount callback

The assign_sounds function looks like this:

    defp assign_sounds(socket) do
  json =
    Jason.encode!(%{
      click: ~p"/audio/button-click.mp3",
      donk: ~p"/audio/button-no-click.mp3",
      win: ~p"/audio/Jingle_Win_00.mp3"
    })

  assign(socket, :sounds, json)
end

  

We create a JSON object attached to :sounds that links the name of a sound to the audio file.

In our LiveView’s index.html.heex template, we link the sounds to the DOM element with the following markup:

    <div
  id="settings"
  phx-hook="AudioMp3"
  data-sounds={@sounds}
>
  ...

  

Here, we assign the AudioMp3 hook to the page using the phx-hook attribute. Then our JSON mapped sound effects object is embedded in the data-sounds attribute. This allows the AudioMp3 JS hook to read in the mapping of sound names to actual files.

Now’s the perfect time to add and register a hook named AudioMp3! Let’s ensure we have that file added now. It reads in the JSON sounds data written to the html data attribute and creates a new Howl object for each audio file entry, pre-loading the audio file in the process.

The other important function of the hook is to let us trigger events either from JavaScript in the browser, or from a LiveView event pushed from the server. Sound effects are triggered by the name we give it, like “click” or “donk”.

To finish out the support for our hook, we’ll configure the app.js file to load and support the AudioMp3 hook.

With all of that in place, we can finally link a button’s click to a sound effect!

Tie a sound to a click event

We’re ready to attach our first sound effect! The first place we should attach our audio effect is to a button click.

Why?

The server cannot trigger a delayed sound event on the page until after the user has interacted with the site. As you can imagine, Ads would be super obnoxious if these rules didn’t exist. In fact, it was past abuses that defined how the features work today.

Here’s an explanation from Mozilla:

Many browsers will ignore any requests made by your game to automatically play audio; instead playback for audio needs to be started by a user-initiated event, such as a click or tap. ~Mozilla Audio for Web Games - Autoplay docs

So, before we start playing audio, we need to tie it to a user-initiated event. Buttons are the perfect solution.

How to trigger a sound effect?

If we had a button on our LiveView that triggered an event on the server named “start-game”, it might look like this:

<.button phx-click="start-game">
  Start Game!
</.button>

The phx-click event "start-game" will fire on the server. Using LiveView’s JS feature, we can add a locally played sound effect and still keep our server-pushed event. It looks like this:

<.button phx-click={
    JS.dispatch("js:play-sound", detail: %{name: "click"})
    |> JS.push("start-game")
  }
>
  Start Game!
</.button>

Nice! We add in a call to JS.dispatch/2 passing the sound event to fire with the details specifying the name of the sound we want to play. The JS.push/3 event then sends along the click event to the server like normal.

Slick!

This will always play the “click” sound effect. What if we want to change the sound based on the some internal state?

Using server logic based on the state of the LiveView, we can change sound effect that’s triggered like this:

    defp assign_start_game_sound(socket) do
  if can_start_game?(socket) do
    assign(socket, :start_game_sound, "click")
  else
    assign(socket, :start_game_sound, "donk")
  end
end

  

Then in our html, we use the sound’s name from our assigns:

<.button phx-click={
    JS.dispatch("js:play-sound", detail: %{name: @start_game_sound})
    |> JS.push("start-game")
  }
>
  Start Game!
</.button>

This makes it easy for our page to change the sound effect played when a button is clicked based on the state of the LiveView (or game). Immediately our page starts communicating state and giving user feedback in a new audible dimension!

Can the server trigger a sound?

Once the user has started interacting with the page and we’ve played sounds connected to those actions, then we’re able to push sounds from the server that aren’t closely tied to a user’s action.

What kinds of sounds? When would we do that?

Here are a few reasons we might want to push a sound effect from the server:

It’s the user’s turn (another player finished)
The user’s report was prepared as is now available
The user won or lost the game based on delayed internal server state

How do we play a sound effect from the server? Like this:

    {:noreply, socket |> push_event("play-sound", %{name: "win"})}

  

At some point, the server determines a sound should be played. That might happen in a handle_info/2 event for instance. The server just needs to push the "play-sound" event, which triggers the "phx:play-sound" event in the AudioMp3 hook, which plays the named sound.

Excellent!

We’ve covered all that’s needed to add sound effects to a LiveView page. With this, we can trigger sound effects when a button is clicked or even have the sound triggered by the server.

Automatically stopping audio

The AudioMp3 hook implements the destroyed() callback, which is called when the attached DOM node is removed. This means when the user navigates away from the page playing sound effects, all sounds are stopped and the audio files are unloaded, cleaning up the resources automatically.

Discussion

Audio adds a new dimension for giving feedback to the user based on the application’s state.

Once we’ve gotten over the hump of supporting audio in our LiveView, it’s quite simple to extend and use it as desired.

Here are a few tips and things to consider as we close:

Don’t be afraid of using sounds - Sounds offer a new dimension for giving user feedback. Explore how it might serve you and your application!
Don’t be annoying - Don’t add sound effects that are obnoxious or don’t communicate anything of value.
WAV files are large, compress them to MP3 files.
Consider where audio can help communicate state or make an experience more immersive.

Happy hacking!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Using LLama.cpp with Elixir and Rustler

2023-12-21T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to use GPUs. Check out how to get started!

Recently I started playing with the idea of using LLama.CPP with Elixir as a NIF. Creating C/C++ Nif’s in Erlang is kind of a project and you need to be especially careful to not cause memory errors bugs. So I found a Rust Wrapper around LLama.cpp and since I have some experience with Rustler I thought I’d give it a go. If this is your first time hearing about Elixir and Rust(ler) you might was to go back and read my initial experiences!

Let’s use LLama.cpp in Elixir and make a new Library!

If you are not familiar with llama.cpp its a project to build a native LLM application that happens to run many models just fine on a Macbook or any regular ole GPU. Today we’re going to do the follow:

Setup a new Elixir Library!
Use the Rust Library rust-llama-cpp with Rustler
Build a proof of concept wrapper in Elixir
And see what we can do!

Scaffolding

Let’s start off like every great Elixir Package with a mix new llama_cpp we won’t be needing a supervisor because this will be a thin shim over the Rust library, rust-llama-cpp. Opening up our mix.exs let’s add our single dep:

    defp deps do
  [
    {:rustler, "~> 0.30.0", runtime: false}
  ]
end

  

Next up installing our deps with mix deps.get, and executing the rustler scaffolding mix rustler.new and following the prompts one by one. Finally adding our final dependency to native/llamacpp/Cargo.toml:

[dependencies]
rustler = "0.30.0"
llama_cpp_rs = {version = "0.3.0", features = ["metal"]}

We should be ready to go! Time to dive into some rust!

A little Rust and Elixir

Looking over the Docs for the rust-llama-cpp library the two core function’s we’ll need to implement are LLama::new and llama.predict new being the constructor for the LLama model and predict handling the actual text prediction. Below is our LlamaCpp module in Elixir with the stubs that Rustler requires:

    defmodule LlamaCpp do
  use Rustler,
    otp_app: :llama_cpp,
    crate: :llamacpp

  def new(_path), do: :erlang.nif_error(:nif_not_loaded)
  def predict(_llama, _query), do: :erlang.nif_error(:nif_not_loaded)
end

  

An example of a typical workflow for exercising this code will look like:

    {:ok, llama} = LlamaCpp.new("path_to_model.gguf")
LlamaCpp.predict(llama, "Write a poem about elixir and rust being a good mix.")
# Very good poem I definitely wrote..

  

Now let’s stub out the Rustler code replacing the body of native/llamacpp/src/lib.rs with:

    use llama_cpp_rs::{
    options::{ModelOptions, PredictOptions},
    LLama,
};
use rustler::Encoder;
use rustler::{Env, LocalPid, NifStruct, ResourceArc, Term};

#[rustler::nif(schedule = "DirtyCpu")]
fn new(path: String) -> Result<LLama, ()> {
  LLama::new(path.into(), ModelOptions::default())
}

#[rustler::nif(schedule = "DirtyCpu")]
fn predict(llama: Llama, query: String) -> String {
  llama.predict(query.into(), PredictOptions::Default).unwrap()
}

rustler::init!(
    "Elixir.LlamaCpp",
    [predict, new]
);

  

Right now if we try running this we will multiple warnings and errors but thats okay we’ve got our scaffolding. You might notice that our Elixir code returns a llama resource that we’re expecting to pass through to our other LlamaCpp.predict function, and while it would be awesome to say this worked “automagically” it does not.

Rustler Resources

We’re going to have to setup a Resource that tells the BEAM virtual machine that this type is something we can hold on to, and that it should clean up when the process goes away. To do this we need to make some changes in our lib.rs

    use rustler::{NifStruct, ResourceArc, Term};

pub struct ExLLamaRef(pub LLama);

#[derive(NifStruct)]
#[module = "LlamaCpp.Model"]
pub struct ExLLama {
    pub resource: ResourceArc<ExLLamaRef>,
}

impl ExLLama {
    pub fn new(llama: LLama) -> Self {
        Self {
            resource: ResourceArc::new(ExLLamaRef::new(llama)),
        }
    }
}

impl ExLLamaRef {
    pub fn new(llama: LLama) -> Self {
        Self(llama)
    }
}

impl Deref for ExLLama {
    type Target = LLama;

    fn deref(&self) -> &Self::Target {
        &self.resource.0
    }
}

unsafe impl Send for ExLLamaRef {}
unsafe impl Sync for ExLLamaRef {}

  

Because we cannot alter the LLama library directly without vendoring we need to wrap it and do the various implementations that the Rustler ResourceArc type requires as a type. I do not fully understand why we need 2 types, a ExLLama and ExLLamaRef type but I was using explorers library as a reference. My understanding is that in order to have the BEAM handle garbage collection you need to wrap your Rust Data in a ResourceArc, which requires that your type implement the Send and Sync protocols to work. The Deref protocol is simply a nice to have for us so we don’t need to dereference our reference type.

The benefit for us is that the BEAM will handle our memory for us and we only need to give it a handle to clean up when it’s done.

Now we can update our functions to look like this:

    fn on_load(env: Env, _info: Term) -> bool {
    rustler::resource!(ExLLamaRef, env);
    true
}

#[rustler::nif(schedule = "DirtyCpu")]
fn new(path: String, model_options: ExModelOptions) -> Result<ExLLama, ()> {
    let model_options = ModelOptions::default();
    let llama = LLama::new(path.into(), &model_options).unwrap();
    Ok(ExLLama::new(llama))
}
// ...

rustler::init!(
    "Elixir.LlamaCpp",
    [predict, new],
    load = on_load
);

  

Notice that we added the on_load callback that registers our ExLLamaRef type, and now that Just Works TM and that we are using the defaults for ModelOptions.

Finally we can use that with our predict function updated like so:

    #[rustler::nif(schedule = "DirtyCpu")]
fn predict(llama: ExLLama, query: String) -> String {
    let predict_options = PredictOptions::default();
    let result = llama.predict(query.into(), predict_options).unwrap();
    result
}

  

Here we accept the ExLLama as a parameter and simply call predict on it, unwrapping the result and returning a string!

Example output

Going back to our original example above let’s try it with a real model and see how it does, please note this is running on a M1 MacBook Pro with 16gb of ram:

    {:ok, llama} = LlamaCpp.new("openzephyrchat.Q4_K_M.gguf")
query = "Write a poem about elixir and rust being a good mix."
LlamaCpp.predict(llama, "GPT4 User: Follow the instructions below to complete the task:\n #{query}<|end_of_turn|>GPT4 Assistant:")
# a bunch of llama-cpp logs then.. and ~10s later
In the land of ancient lore, where myths entwine and roam,A tale of magic potion whispers and hums.\nElixir and rust, an unlikely pair,\nConjure up a story that leaves us in awe.\n\nBorn from the heart of alchemy's embrace,\nThe elixir, radiant as daylight's face,\nPromises youthful vitality and grace,\nA fountain of immortality to chase.\n\nYet rust, the nemesis of fine metal sheen,\nInfiltrates"

  

In the land of ancient lore, where myths entwine and roam, A tale of magic potion whispers and hums. Elixir and rust, an unlikely pair, Conjure up a story that leaves us in awe.

Born from the heart of alchemy’s embrace, The elixir, radiant as daylight’s face, Promises youthful vitality and grace, A fountain of immortality to chase.\n\nYet rust, the nemesis of fine metal sheen, Infiltrates"

I don’t think it picked up on our programming language, let’s try again with: “Write a poem about the programming languages elixir and rust being a good mix.”

In the world of code, where ones and zeros dance, There’s Elixir and Rust, a pair that romances. Their union is strong, like a fortress of steel, In harmony they stand, together they heal.

Elixir, with its elegance and grace, A functional language, in the Erlang base. Concurrency and fault tolerance are its strengths, Like an actor system, it never takes rests. Rust, a systems programming powerhouse, Safe and fast, like a stealthy fox. Zero-cost

Conclusion

So as you can imagine this is only the tip of the iceberg here for what’s possible and the rust-llama-cpp library has more API to implement. I’ve started work on that here llama_cpp_ex and as always contributions are very welcome! I hope today we got a taste of using long living structures in Rustler and how we might work with them, As well as had a little bit of fun with a LLM.

If you want to run these models with LLama.cpp on a Fly GPU you can now do that too!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Real World ™ Machine Learning on Fly GPU's

2023-12-08T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to use GPUs. Check out how to get started!

We live in truly a time of wonders; every single week, a new demo drops with a mind glimpse of what our incredible AI future might be. From generating illustrations, front-end apps, or very convincing text, however…

If we’re being honest, those demos, while incredible, are difficult to even imagine putting into a real-world application. We’re not all living in a world where we can send our customer’s personal data off to a SaaS. Nor are all of us enthusiastic to lock into another developer SaaS platform. Most of us simply want to help our customers solve their problems using our own tools.

The good news is we have options. Models are being trained and released under open-source licenses on HuggingFace. Elixir is expanding its capability and reach far past the world of the Web and into Machine Learning. Now Fly.io is offering GPU’s to anyone who is interested, it is as simple as choosing the correct Dockerfile and a flyctl command we can have some of the world’s most powerful devices working for us. While also running in the same data centers as our applications, close to our customers, starting and stopping when we need them.

Search all of HexDocs, again

Let’s see how we can use a GPU in a real world project. In a previous post, we went on an adventure, collecting all of the Hexdocs and building a SQLite Database with them. Using the built-in SQLite FTS5 Search Index and culminating in a kinda useful website. This actually started a ball rolling to get a more useful and universal docs search going that’s available to all of Introduce search across all of HexDocs. So let’s try our hand at building our own Semantic Search for all of Hex Docs!

In this post, we will build off that existing database and build a new search index. We’ll be doing the following:

Using a Fly.io GPU with Livebook
Using Bumblebee.Text.TextEmbedding.text_embedding to generate a vector based on the documents.
Index them using the hnswlib index.
Then query the index using both.

Setup

We’ll be following the GPU Quickstart guide, and because we’re not starting from an existing Phoenix project, we’re going to start our project a little differently in an empty directory:

fly apps create --region iad

Create/modify the fly.toml:

app = "name"
primary_region = "iad"
vm.size = "a100-80gb"

# Use a volume to store LLMs or any big file that doesn't fit in a Docker image
[[mounts]]
  source = "data"
  destination = "/data"
  initial_size = "40gb"

[build]
  image = "ghcr.io/livebook-dev/livebook:latest-cuda12.1"

[env]
  ELIXIR_ERL_OPTIONS = "-proto_dist inet6_tcp +sssdio 128"
  LIVEBOOK_DATA_PATH = "/data"
  LIVEBOOK_HOME = "/data"
  LIVEBOOK_IP = "::"
  LIVEBOOK_ROOT_PATH = "/data"
  BUMBLEBEE_CACHE_DIR="/data/cache/bumblebee"
  XLA_CACHE_DIR="/data/cache/xla"
  XLA_TARGET="cuda120"
  PORT = "8080"

[http_service]
  internal_port = 8080
  force_https = true
  auto_stop_machines =false
  auto_start_machines = false
  min_machines_running = 1
  processes = ["app"]

Take note of some newish options specifically the vm.size where we call out the GPU size we’re interested in and initial_size on our volume. We’re also referencing the official Livebook CUDA image.

We will need to set a secret for the password

fly secrets set LIVEBOOK_PASSWORD="very secret password"

Finally fly deploy and that’s it! When it’s done deploying, we’ll have a fully working Livebook, and we can visit the URL and login with our very secret password!

How do we use a GPU for text search?

When we normally use one of these Large Language Models (LLM), our experience is essentially converse(model, "Hello AI") but what’s actually happening is as follows:

Your text is encoded into a list of floats, otherwise known as an Embedding Vector…
… then “passed through” the model, which outputs another vector
… which is decoded into text.

Those vectors represent a multi-dimensional vector space, and you can compare the cosine of the angle between two N-dimensional vectors to see how similar they are.

This sounds complex, but by the end of this post, we’ll have done it. We are going to use the text_embedding function from Bumblebee with a good model for text retrieval, pipe that into a tool for indexing vectors by cosine similarity, and ideally, it gives a better search result than SQLite FTS5!

Choosing a model is not a small task; based on a very scientific search, I found this chart that claims to test and compare models, and the bge-large model seems to be compatible with Bumblebee and pretty high on the chart, so let’s roll with it! If you are playing along at home, the bge-small model also seems to do pretty good!

Let’s open a new Livebook and get started!

    Mix.install([
  {:bumblebee, "0.4.2"},
  {:exla, "0.6.4"},
  {:hnswlib, "0.1.4"}
])

Application.put_env(:exla, :clients,
  cuda: [
    platform: :cuda,
    lazy_transfers: :never
  ]
)

Nx.global_default_backend(EXLA.Backend)

  

Adding a new cell for our model.

    repo = {:hf, "BAAI/bge-large-en"}
{:ok, model_info} = Bumblebee.load_model(repo, architecture: :base)
{:ok, tokenizer} = Bumblebee.load_tokenizer(repo)

  

Here we download our model and load it up. To keep this post from becoming too long, we’re skipping the part where we massage the data into a clean format, I simply queried it from the SQLite database we built in the last post and ended up with something like this:

    docs = [
  {"string-id", "doc text and title....sometimes very long"},
  ...
]

  

There is one small issue here, and that’s that our doc text can sometimes be larger than the max sequence_length as defined in the model of 512. If we don’t chunk up our document when we calculate the embedding, it will truncate anything greater than 512.

    docs =
  Enum.flat_map(docs, fn {id, doc} ->
    doc
    |> String.codepoints()
    |> Stream.chunk_every(512)
    |> Stream.with_index()
    |> Enum.map(fn {chunk, chunk_id} ->
      doc = Enum.join(chunk)
      {"#{id}-#{chunk_id}", doc}
    end)
  end)

  

This chunks each string into 512 segments and then gives it a new id of id-N to help us keep the order of everything later.

Building our Index

Now we have everything we need to finally calculate our embeddings and create our HSNW index.

    dim = 1024
space = :cosine
batch_size = 64
sequence_length = 512

  

Here we have some variables,

The embedding (for the large model) will produce a vector of dimension 1024, so our index needs that size.
We want to use :cosine similarity when building our index
Our Fly.io GPU can do a batch_size of 64 embeddings at once, a smaller machine may need to tune that down to as low as 1; this requires experimentation where you start at 1 and increase by powers of two until XLA complains about running out of memory in the next steps.
The sequence_length is defined by the model as the max number of tokens it was trained on.

    serving =
  Bumblebee.Text.TextEmbedding.text_embedding(model_info, tokenizer,
    defn_options: [compiler: EXLA, lazy_transfers: :never],
    output_attribute: :hidden_state,
    output_pool: :mean_pooling,
    compile: [sequence_length: sequence_length, batch_size: batch_size]
  )

Kino.start_child(
  {Nx.Serving, serving: serving, name: BGE, batch_size: batch_size, batch_timeout: 0}
)

  

This will set up a process to handle our text embedding requests. Most of these options are optimizations I picked up from the Erlref Slack #machine-learning channel, hat-tip to them!

The only key bit for us is the name BGE which we will reference as the process to send requests to. Let’s start with the entire indexing function, and I’ll break it down bit by bit afterward.

    max_elements = Enum.count(docs)
{:ok, index} = HNSWLib.Index.new(space, dim, max_elements)

indexed_docs =
  docs
  |> Stream.chunk_every(batch_size)
  |> Stream.flat_map(fn chunk ->
    chunks = Enum.map(chunk, fn {_id, doc} -> doc end)
    embeddings = Nx.Serving.batched_run(BGE, chunks)
    Enum.zip(chunk, embeddings)
    |> Enum.map(fn {{id, doc}, %{embedding: t}} ->
      {id, doc, t}
    end)
  end)
  |> Stream.with_index()
  |> Stream.map(fn {{composite_id, doc, embedding}, i} ->
    HNSWLib.Index.add_items(index, embedding, ids: [i])
    IO.puts("#{Float.round(i / max_elements * 100, 3)}%")
    {i, {composite_id, doc}}
  end)
  |> Enum.into(%{})

HNSWLib.Index.save_index(index, "/data/bge-index-large.bin")
File.write!("/data/indexed_docs.bin", :erlang.term_to_binary(indexed_docs))

  

First up, we set the HNSWLib.Index and we chunk the doc’s list into the same size as our batch_size using Stream.chunk_every.

ASIDE: This is a very good example of when to use a Stream over Enum. In this case docs is not small, possibly a hundred or more megabytes, if we used Enum it would create a copy between every step.

We’ll flat_map over the chunks sending each chunk to Nx.Serving.batched_run(BGE, chunks). We reference our process by name, BGE and the result is a list of maps with key embedding.

We need to associate those back with the original data so that when we index it, we can search for it, so we zip the embeddings up with the original docs and continue building our Stream pipeline.

When using HSNWLib you can give it an “id” for a document, but it’s limited to integers, so by using Stream.with_index we give ourselves an incrementing integer for each individual doc. We add our embedding to the index with its id and return the index and doc data.

Finally, because we’ll want to quickly grab a document by its id we dump the final results into a map. We’re also adding a logger to show progress because to run this on our very crazy GPU still takes about an hour.

As a result, I save the index and my values to disk just in case I do something dumb and lose my Livebook context!

Querying!

Now to query, we’ll generate the embedding for our query, use it with HNSWLib.Index.query, and finally, we’ll grab our doc from the indexed_docs map we built above and see if we did a good job!

This model recommends a prompt when searching:

    prompt = "Represent this question for searching relevant passages:"

  

We’ll append our query to that and send it along to our same Nx.Serving setup as before and then querying HNSQLib:

    query = "#{prompt} How do I make an inner join using ecto?"
[%{embedding: query}] = Nx.Serving.batched_run(BGE, [query])
{:ok, results, weights} = HNSWLib.Index.knn_query(index, query, k: 3)

  

And finally, we need to massage the results that HNSWLib gives back:

    results = Nx.to_list(results) |> hd
weights = Nx.to_list(weights) |> hd
unindexed_docs = Map.values(indexed_docs)

for {w, id} <- Enum.zip(weights, results) do
  {id, _doc} = result_doc = Map.get(indexed_docs, id)
  [id, _] = String.split(id, "-")
  to_find = "#{id}-"
  doc =
    unindexed_docs
      |> Enum.filter(fn {id, _doc} -> String.starts_with?(id, to_find) end)
      |> Enum.sort_by(fn {id, _} -> id end)
      |> Enum.map(fn {_id, doc} -> doc end)
      |> Enum.join()

  IO.puts("#{w}: #{doc}")
end

  

For each result, we get the doc using our index, find all docs with similar id-prefixes, sort them, and finally join them into a single doc.

Results:

0.09391725063323975: Joining with fragments - Ecto.Query...
0.09523665904998779: Ecto.Query.join/5 A join query ..
0.09840273857116699: locus - Documentation Supported F...

Not bad! It found two relevant Ecto Docs and a third document of questionable relevance! When running the query, it took under 100ms which is well in the realm of autocomplete-level performance.

Wrap up

I would be lying to say any of this was simple. I had to learn about GPU’s, fight Nvidia packages, find and diagnose bugs in multiple Elixir packages, use trial and error to find the correct optimized options for XLA, and also develop even more patience because this stuff takes forever to run. Here are some of my top tips for less pain:

Use Nvidia’s Dockerfiles. Livebook’s Dockerfile is based on Nvidia’s, and that seems to just work. It is pure pain to deviate from this path.
Join the Erlref Slack #machine-learning channel. The more you push Bumblebee/Nx, the more you will want advice from the authors of those packages, and vice versa; they love getting feedback!
Write results to disk. I lost a couple of very long-running experiments because I didn’t write the results to disk. Everything about this process gobbles up memory, and you can quickly OOM and lose everything. Or you know you can have a syntax error that takes 11hrs to be found.

Getting this ready for production is as simple as spinning up an Elixir server that loads the model, index, and joins our production cluster, allowing our web server to call it.

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Abusing LiveView's new Async Assigns Feature

2023-11-09T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Previously, in Star-Crossed LiveView Processes, we explored using Elixir’s linked processes to connect a Task and a LiveView together. This required doing some extra work like trapping exits and handling received EXIT messages, but, in the end it work just like we wanted.

Here, we’ll revisit solving the same problem but using Phoenix LiveView’s newly built-in async operations and see how it compares. Chris McCord talked about this in his ElixirConf 2023 keynote video.

Before we go further, just know you can grab the finished code for this article in this gist.

We’ll briefly review the problem we are solving that was tackled in Star-Crossed LiveView Processes. If you’re already clear on what we’re doing here, jump ahead to Starting the Task.

Recap: What problem are we solving?

We want our LiveView to launch an asynchronous process. With LiveView v0.20, some incredible new async operation features were added. This provides new building blocks for our LiveView to execute work in a separate but linked process.

The primary use case for these async operations is described in the docs this way:

It allows the user to get a working UI quickly while the system fetches some data in the background or talks to an external service, without blocking the render or event handling. For async work, you also typically need to handle the different states of the async operation, such as loading, error, and the successful result. You also want to catch any errors or exits and translate it to a meaningful update in the UI rather than crashing the user experience.

This works really well for most use cases. Think of loading a LiveView then starting a separate process to go fetch some data that will be immediately displayed once it’s ready.

In our use case, we are deviating from the happy path. From the docs, note the focus is on wanting the final “successful result”. Here’s how our use case differs:

We’re not starting the process when the LiveView mounts. It happens as the user interacts with the LiveView.
Rather than wait for the final result, we want a real-time stream of messages.

Here’s a visualization for the focus being more on the messages sent back while the process is running rather than the final result.

Before we jump into the code, let’s define a bit more about how we want our application to behave.

Goals for application behavior

Let’s outline how the application should behave, particularly around the async process and our LiveView.

The LiveView process remains unblocked. Our blocking external calls should happen in a separate process. Let’s keep the UI buttery smooth.
When the LiveView process goes away, the other process should too. Because the worker process only exists to fetch and provide data to the LiveView, we want the async process to stop if the user closes the page or navigates away.
When the async process crashes, it should NOT kill our LiveView. We expect an async process talking to an external API will fail sometimes. We do not want that to crash the UI for the user.
Ability to cancel a running worker process. We want the ability to cancel a running async worker from the LiveView.
We care about the side-effects, not the final result. Think of a ChatGPT-like text steam, that’s what we want too. We prefer to show the user what we have now rather than hiding received data until it’s fully complete. These small chunks of data are still meaningful for the user and are the side-effects we’re interested in.

All right! That may seem like a hefty set of requirements, but it gives us the user experience we want and now using async assigns makes it even easier that it was before!

Here is a visual example of the behavior we want.

With an idea of what we’re building, let’s dive in and see how we start the async task!

Starting the task

In this article we’re more concerned with how the LiveView and async task interact and we’re not focusing on the actual work the async operation is doing.

When the user clicks the “Start” button, the handle_event clears the messages and starts the Task.

    def handle_event("start", _params, socket) do
  socket =
    socket
    |> assign(:messages, [])
    |> start_test_task()

  {:noreply, socket}
end

  

In our start_test_task function, we use the new start_async/3 which takes an anonymous function that is executed asynchronously.

    def start_test_task(socket) do
  live_view_pid = self()

  socket
  |> assign(:async_result, AsyncResult.loading())
  |> start_async(:running_task, fn ->
    # the code to run async
    Enum.each(1..5, fn n ->
      Process.sleep(1_000)
      IO.puts("SENDING ASYNC TASK MESSAGE #{n}")
      send(live_view_pid, {:task_message, "Async work chunk #{n}"})
    end)
    # return a small, controlled value
    :ok
  end)
end

  

The first thing we do is call self() and assign the pid of the LiveView process to a variable that can be referenced in our async function. This passes the pid of the LiveView via a closure to our async function.

Before we start the work, we execute the AsyncResult.loading() function which tells LiveView that our :async_result (this could be named anything you want) is now in a loading state.

The “work” being done in our function is looping 5 times, sleeping for 1 second, then sending a message to the LiveView process about the chunk of work we completed.

When the function finishes, we’ll be notified and we’ll exit the loading state. That comes later.

LiveView’s new `handle_async/3` callback

LiveView v0.20 introduced a new handle_async/3 callback function that fires when the result of a start_async/3 operation is available.

We’ll use this to update our UI when the async process either:

completes successfully
completes with an error
the task process crashes

Pattern matching helps us tell the difference between these three result states.

Something worth pointing out is that the first argument to the callback is the name we gave our task when we started it. This means that we can start multiple concurrent async tasks and deal with them all uniquely if we want. That’s pretty cool!

Let’s see what a “success” callback looks like next.

Successful Completion

When our async function completes successfully, the handle_async/3 callback is fired in our LiveView and we use pattern matching to determine that it succeeded.

If you recall, the result of our async function was :ok because we aren’t waiting for the final result in our situation. That :ok is what’s coming through in our handle_async function.

    def handle_async(:running_task, {:ok, :ok = _success_result}, socket) do
  socket =
    socket
    |> put_flash(:info, "Completed!")
    |> assign(:async_result, AsyncResult.ok(%AsyncResult{}, :ok))

  {:noreply, socket}
end

  

The handle_async function receives {:ok, function_result}. So in our case, a success is {:ok, :ok}. Yes, that looks odd, but it is valid. 🙂

In the callback, we do two things.

Display a flash message that it completed successfully.
Clear the “loading” state by updating the AsyncResult assigned to :async_result.

We use AsyncResult.ok/2 to record that the async process succeeded. We’re passing in two values:

%AsyncResult{} - clears our loading state and resets the :async_result value for being called again.
:ok - This would be the result of our function if we were using it that way. Since we don’t have a relevant value to store, we just store :ok.

Let’s see how to detect a failure.

Detecting when the function fails

When the anonymous function passed to start_async/3 returns an error like {:error, data}, it’s actually still a successful result from our function. “Successful” meaning the function didn’t explode but actually returned something.

In our situation, we could send a message about a failure to the LiveView, or we could let the function return some error data.

If we chose to return the error state and data from our function, it is still received through the handle_async/3 callback. Our pattern match might look something like this:

    def handle_async(:running_task, {:ok, {:error, reason}}, socket) do
  socket =
    socket
    |> put_flash(:error, reason)
    |> assign(:async_result, AsyncResult.failed(%AsyncResult{}, reason))

  {:noreply, socket}
end

  

Note we’re still matching on the outer {:ok, result} tuple. This is because the function didn’t blow up. But with this error information, we can display a flash message with the reason.

Next, we use the AsyncResult.failed/2 function to track that it failed. Again, we reset the AsyncResult to clear the loading state.

Detecting when the function explodes

We’ve touched on the idea that the function isn’t exploding when we detect the success and failure states. Sometimes functions DO explode!

Why might a function explode?

an exception is raised
the async process dies (an exception, DB failure, OOM, network connection closed, etc)

In either case, it’s a total failure, but we still want to handle it because one of our goals is that a failure in the task doesn’t crash our LiveView UI.

A task that died still comes through the handle_async/3 callback. Here’s what the pattern match looks like:

    def handle_async(:running_task, {:exit, reason}, socket) do
  socket =
    socket
    |> put_flash(:error, "Task failed: #{inspect(reason)}")
    |> assign(:async_result, %AsyncResult{})

  {:noreply, socket}
end

  

The pattern is {:exit, reason}. This tells us the processes executing our function died, or exited.

Here we display a flash message about the error with whatever information we have.

Finally, we clear our :async_result back to a fresh and shiny blank %AsyncResult{} . This clears the loading state so we can try again if desired.

All that’s left to cover is, “how do we cancel a running task?” Let’s explore that next.

Cancelling the task

One of our goals was to be able to cancel the running async process. How do we do that?

To review, when we called start_async/3, we assigned it to :running_task in our LiveView’s assigns. We can use this to cancel the process.

Also, we’re using :async_result to track the %AsyncResult{} struct, specifically to know when we’re loading or not.

In our UI, let’s conditionally display a “Start” and “Cancel” button using this “loading” state. The following markup displays a “Start” button when no async process is running and the “Cancel” button when @async_result.loading.

    <div class="...">
  <.button :if={!@async_result.loading} phx-click="start">Start</.button>
  <.button :if={@async_result.loading} phx-click="cancel">Cancel</.button>
</div>

  

Now our “Cancel” button only shows up when an async process is running. That was easy!

Next we’ll handle when the “Cancel” button is clicked.

    def handle_event("cancel", _params, socket) do
  socket =
    socket
    |> cancel_async(:running_task)
    |> assign(:async_result, %AsyncResult{})
    |> put_flash(:info, "Cancelled")

  {:noreply, socket}
end

  

LiveView introduced cancel_async/3 just for this purpose!

We call cancel_async using the name of the key stored in our assigns from our start_async call.

We also clear the :async_result to reset the “loading” state. Now our UI correctly reflects that there is no longer a process running and we could start it again if we wanted.

Finally, we add a flash message to display that it was cancelled.

Nice! All cancelled and cleaned up!

We succeeded in:

starting an async process from our LiveView
tracking and displaying when the process is running (aka “loading”)
handling successful completion
handling async task failures
handling when an async task crashes
canceling a running task

We did all that using the Phoenix LiveView’s new async operations.

Discussion

Be sure to check out the full gist for putting all the pieces together!

Our situation, where we aren’t waiting for the final result, is outside the normal flow but Phoenix LiveView’s async assigns still works for us. Let’s note what was different.

We aren’t using assign_async

We aren’t using the assign_async/3 function which is what the whole feature is named for! That function works best when we’re loading something in our LiveView’s mount.

Instead, we use start_async/3 which gives us more granular control for starting and stopping our async task.

We still use `%AsyncResult{}` even when we don’t care about the result

Yup. We still used %AsyncResult{} even though we aren’t using it to store our result. It is still a handy way to track our loading state and if our operation was successful or if it failed.

What we gain from async assigns

Turns out the new async operations are versatile and powerful enough to let us solve a variety of async problems. The feature wraps up the boilerplate needed for:

starting async tasks
linking processes
trapping exits
cancelling tasks
tracking when we’re loading
and more!

All in all, this is a very welcome addition to Phoenix LiveView! Most people will probably use it the prescribed way by fetching async data in a LiveView on mount. But it should give us greater confidence that it’s also flexible enough when our project’s needs take us off the async operations happy path.

Have you given the new features a spin yet?

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Elixir and Phoenix can do it all!

2023-10-26T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

As an Elixir/Phoenix developer going on 10 years it is very easy to take for granted everything that Elixir and Phoenix can do for us. So I wanted to take a step back and list all of the stuff that we get for free when we choose Elixir and Phoenix.

My goal with this post is dump an list of information and links and let you choose your own path. If you are unfamiliar with any of these topics or Elixir and Phoenix please click any of these links and take a peak, I think you might be surprised at the quality and depth of what you find!

The inspiration for this post was a Saša Jurić re-tweet where he shared the following slide from his GOTO Conference Talk.

This image enumerates some of the technical requirements that the Erlang Virtual Machine can replace in your tech stack. Not to say that these other projects can’t do these things well, they can and do in our own stack here at Fly.io. The point is that in those ecosystems you have to solve these problems in the first place, when reviewing this article José Valim said it best:

removing the problem altogether instead of solving the problem

The BEAM

This above list doesn’t even cover the built in support for incredible bits like:

Actor Process Model, meaning due to concurrency you have a Process and it sends messages to other processes. No Shared memory. Fully managed by the BEAM’s built in scheduler. Millions of processes are no issue. Fearless concurrency!
Distribution: meaning two servers can communicate seamlessly and automatically. It just works and is incredible.
RPC: Calling a function or process remotely is as easy as calling a local one, no need to concern ourselves with connections or serialization or deserialization. This is one of the many reasons Fly is so committed to using Elixir and Phoenix, the wireguard based network and globally hosting makes using this trivial and desirable.
Parallel Garbage Collection, meaning the VM is Soft Real Time, minimal pausing!
Emphasis on low resource usage in general
Built in UDP/TCP/SSL server support.
Built in Key Value Store ETS.
Tooling for tracing and observing your code.
37 Years of constant development and production use.

The beauty of this list is that if Elixir and Phoenix team have done our Job right you don’t need to think about any of this at all. It just works for you and lets you stand on the shoulders of this giant.

Elixir

If we go higher level the Elixir programming language provides us even more as developers:

A full dependency management and build tool with mix and Hex.
With modern tooling built in:
- Testing
- Documentation
- Formatting
Language with a clean, documented and fully exampled Standard Library. Its remarkable how little cruft there is in this Language. Go into any other language to find random functions or modules that are marked do not use, or without examples or documentation. Not the case for Elixir.
Modern Documentation:
- Built in Cheatsheets
- Built in Custom Markdown
- Links to source
- Search
- Support for Mermaid, Math.js, Vega-Lite and more!
Modern language features like:
- Protocols enabling high level collections and functions like Enum, Streams, and Access
- Stream, enabling lazy and performant data processing
- Full featured Date/Time/DateTime/Calendar modules.
- Task for trivial parallel processing.
- Agent for trivial shared state management.
- Config for environment specific configuration
- Registry for local, decentralized and scalable key-value process storage.

This is just a subset of what Elixir the Language provides. It doesn’t include the deep catalog of libraries within package registry Hex enabling so much more.

Some examples from Hex:

Machine Learning with Bumblebee with HuggingFace integration and it works with batching and distribution globally.
Math/Science comparable to NumPy using Scholar.
Complex distributed big data processing using GenStage or Broadway
Embedded and IoT Programming using Nerves.
Stream-Data for property and generative based testing.
Sobelow for security and static Analysis.
Oban for a Worker/Job Queue implementation.
Req for high level HTTP Clients.
Plug for HTTP Servers.
Bandit for pure Elixir HTTP1/2 Server!
LiveBook for Juypter like workbooks with Elixir!
Membrane for video stream processing.
Decimal for arbitrary precision decimal arithmetic.
Jason for highly performant JSON Encoding/Decoding.
Image for image manipulation.
CLDR for maybe the most complete Internationalization and Locale libraries outside of maybe the Web Browser. Numbers, List, Units, Date\Time\DateTimes, Collation, Territories, and more.
Rustler and Zigler for truly simple Rust and Zig FFI support, for when you need to drop down and bash bits.

There is very little that the Elixir Ecosystem hasn’t tackled!

Phoenix and LiveView

The fun doesn’t stop there because Phoenix simply builds on all of his incredible tooling to provide us with:

A fully featured HTTP library with
- Advanced Routing
- Plug-able steps
- Built in HTML and JSON handling
- With HTTPS with sane security defaults
- Secure Cookie Sessions
- Secure Tokens
- Controller Actions for MVC Style apps.
- Built in Internationalization Tooling
Websockets with
- Multiplexing via Channels
- JavaScript library that handles backoff and common error modes
- Serialization
- Sessions
- Scalable to millions of clients.
- PubSubFully Distributed
- With optimizations for “hot paths”
- Presence Monitoring and tracking who is in a channel
- Fully Distributed
- Built using CRDT’s
- JavaScript support
Development Tooling:
- Live Code Reloading
- Helpful and actionable Errors
- Built in End to End Testing. (No Chromedriver needed)
JavaScript
- Bundling/Minification/Integrity/Cache Busting
  - Using esbuild that is installed for you
- NPM support
- Custom JS is trivial
CSS via Tailwind
- Bundling/Minification/Integrity/Cache Busting
  - Using tailwind that is installed for you
- Custom CSS is trivial
LiveView
- Simple, server rendered Live HTML and CSS, in pure Elixir.
- HTML and Html Attribute Compile Time Verification
- Highly Optimized for the modern web
- Debug tooling for Dev
- JS Interop is Trivial
- Latency Simulation
- Single BEAM Process per connection
- Trivial File Uploads to disk or the cloud.
Databases with Ecto
- Postgres/MySQL/Sqlite/and more
- Full Query DSL with Raw SQL escape hatch
- Migrations Up/Down/Schema dump
- Full Schema and Change Validation
- Transaction Support
- Multi-step Queries/Transactions
Full Authorization/Authentication generator
- Register, Login, Forgot Password, Email Verification and Sessions
- LiveView or Server HTML
Email Rendering and Sending
Built in Metrics Tracking
Single Line to add a full Metrics Dashboard
Dockerfile generator

Just going through this list there is everything you’d need to start a company or build a website to solve nearly any problem. Coupled with the BEAM’s ability to scale from the smallest server to a globally distributed network with millions of customers on every continent! I may sound a little breathless but I am out of a breath just collating all of these links.

Wrap up

Stepping back it’s incredible that a small team is able to accomplish so much. Every one of Phoenix’s incredible features is built on the shoulders of the fantastic Elixir project underneath. Further backed by 37 years of constant development by the Erlang project.

If you haven’t looked into Phoenix or Elixir I urge you to explore any one of the above links that catch your eye. Installing and setting up Elixir has never been easier with LiveBook or Natively!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

ChatGPT doesn't know what day it is?

2023-10-23T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

It turns out ChatGPT doesn’t know anything about the date or day of the week! When that’s needed for your application, how can we solve it? Fortunately, Phoenix LiveView and hooks make it easy to get the user’s timezone from the browser for localizing the date information before we give it to ChatGPT so it has the information it needs. Let’s go!

If you ask ChatGPT what day it is, it doesn’t know.

Most of the time, today’s date isn’t relevant. However, while building my personal AI Fitness Trainer, it did matter. The AI Trainer designed my weekly exercise plan and when I asked, “What is my workout today?” it hallucinated and responded, “Since today is Monday, according to your plan…” but it was Friday.

Clearly not helpful.

Sometimes we do need ChatGPT or another LLM (Large Language Model) to be date aware.

Thankfully, this is easy information for us to provide as context to a conversation when it’s relevant to our application.

Using the Elixir LangChain library, we can easily build prompt templates that make it easy to assemble extra information, like today’s date, into our prompts.

Remember: ChatGPT and other LLMs are text based. They specialize in being good at predicting the next text you want to see. They are not good at math or date algebra.

The key for us is to provide it with the needed date information in a good text format. Avoid giving it ambiguous dates like 10/11/23 which could be October 11th or November 10th based on a date format we didn’t provide.

Also, we can’t count on ChatGPT to arrive at the correct day of the week based on a date. Again, ChatGPT is predicting text, not running date algorithms and taking leap years into account, because, well, it doesn’t!

Thankfully, this is also easy data for us to provide! We’ll just give ChatGPT the current day of the week.

The text we’ll send to the LLM will look like this:

Today is Thursday, 2023-10-19

The date format is less ambiguous and we are also explicit about the day of the week.

The code to do this an application that uses the Elixir LangChain library looks like this:

NOTE: Using a PromptTemplate for a single statement is overkill, but it’s really helpful when we’re bringing more information together.

    current_date_template = LangChain.PromptTemplate.from_template!(~S|
Today is <%= @today %>|)

LangChain.PromptTemplate.to_message!(current_date_template, %{
  today: DateTime.utc_now |> Calendar.strftime("%A, %Y-%m-%d")
})

  

This returns a Message struct that we’ll send to the LLM later. The Message’s contents have the following text:

    "\nToday is Thursday, 2023-10-19"

  

Yay! When we include this text, ChatGPT knows the day and date!

Wait, the date is in the wrong timezone!

The date was generated using DateTime.utc_now/0. This means that an evening workout in my US Mountain timezone will report the date as tomorrow!

Well dang. We were so close!

In order to get the correct local date, we need to know the user’s timezone. We might be tempted to use the machine’s system clock. That works for a local development machine, but it fails when the app is deployed to a server. The machine will certainly be in the wrong timezone for most users and it may not even be setup for a timezone at all.

How can we get the timezone? We could ask the user, but that’s an annoying UX.

There is an easier way.

Let the browser tell me it’s timezone

We won’t ask the user, we’ll ask the user’s browser!

The browser knows what locale and timezone it’s running in. For applications that work best when we know the user’s timezone, then this is a handy approach with a good UX.

With LiveView, we can create a JavaScript LiveView hook that fires when the LiveView page mounts. It’s simple JavaScript that fetches the device’s timezone and pushes it to the server.

browser_timezone.js

    export const hooks = {
  BrowserTimezone: {
    mounted() {
      let tz = Intl.DateTimeFormat().resolvedOptions().timeZone
      this.pushEvent("browser-timezone", {timezone: tz})
    }
  }
}

  

We link this hook to our page in the LiveView template like this:

    <main id="..." phx-hook="BrowserTimezone">

  

Now, when the LiveView page mounts, the hook pushes up the browser’s timezone information in a handle_event call in the server. This is what that looks like:

    def handle_event("browser-timezone", %{"timezone" => timezone}, socket) do
  # do something with the timezone

  {:noreply, socket}
end

At this point we do something with the timezone information. We could assign it to the LiveView socket, store it with the user’s DB record, or something else.

In the linked demo project, it is written to the DB for the user.

Great! Now the server knows the user’s timezone!

Lastly, we’ll update the way we generate the date text sent to ChatGPT to be relative to the user’s timezone.

    DateTime.now!(current_user.timezone)

  

Excellent!

Now our ChatGPT sessions can get the day of week and date correct every time!

Wrap up

We covered how ChatGPT doesn’t know the date and instead specializes in predicting the text we want to see. We learned how we can easily tell ChatGPT (and other LLMs) the date information it needs when it’s relevant to our application.

We also covered how to get the user’s timezone from their browser. This can be helpful for many applications that have nothing to do with ChatGPT. It’s a handy trick and LiveView + Hooks make it easy!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Show me the code!

A demo project is available with all the code discussed here:

agent_chat_live - Directory with LiveView and template. Includes subdirectory with supporting modules.
- index.ex - Agent LiveView module where events are handled and timezone is used in ChatGPT message.
browser_timezone.js - The hook definition.
app.js - The hook is registered in app.js.

Created my Personal AI Fitness Trainer in 2 Days

2023-10-16T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

This post introduces how I created an AI Personal Fitness Trainer named “Max” that acts in a way that works best for me. In 2 days I had a new workout buddy and it has reignited my fitness excitement! I quickly had a weekly fitness program that works towards my goals and a trainer who records my workout information for me. Check out a video DEMO of it working and, most importantly, check out the demo Elixir project to get and customize your own fitness buddy!

First, to know what we’re talking about, get an overview of how it works and to see the AI Fitness Trainer in action, check out the video.

Want to dig in right away? You can find the Elixir LangChain Demo project here.

DEMO NOTE

The initial released DEMO project was from a weekend spike and more refactoring, tuning and cleanup would improve it. But that’s partly the point, in a short time, we can create tools that directly impact our lives and make a positive difference. 💪

What is a conversational agent?

We’re talking about an “Agent” here. An agent is described as:

Agent: a language model is used as a reasoning engine to determine which actions to take and in which order.

There are different kinds of agents, but for our AI Fitness Trainer, we’re using a “conversational agent”. That means the user interacts directly with it and most of the agent’s responses are shown to the user.

Let’s describe how this works in practice. We’re using the Elixir LangChain library to talk to ChatGPT 4 through an API. To run this locally, you’ll need your own OpenAI API key.

With an API key, the Elixir LangChain library helps with the following:

Wrapping our custom context around an LLM (Large Language Model) like ChatGPT.
Exposing “functions” to the LLM that it can call in our application.
Template user data and instructions into a single user message sent to the LLM.
Talking to ChatGPT and processing the results.

The following diagram gives an overview of the pieces:

Make note of the context wrapped around ChatGPT and the functions shared with the agent that have controlled access to the application’s data.

Neither the Context nor the available functions are visible to the user. Let’s focus a bit more on those “hidden” pieces next.

Limit what the user sees

We can think of our application as having two separate but linked conversations going on a the same time.

The user has a conversation with our app.
Our application has a conversation directly with the LLM.

There’s a lot of overlap in these conversations, but there is more to the conversation with the LLM going on behind the scenes than what the user sees.

It starts with the Context we define around ChatGPT for how it should behave. The user doesn’t see any of those instructions. We frequently put those instructions in a system message, instructing the system how to act.

Then, if and when the LLM decides to execute a function from our app, we don’t display the details of the request or our application’s response to the user. Handling those function requests is something the Elixir LangChain library wraps up for us in a Function.

There may be other messages that our app needs for talking to the LLM that we don’t want displayed to the user. For this reason, we’ll create a ChatMessage struct for displaying messages in the UI, and internally, we’ll use a collection of LangChain.Message structs for talking with the LLM. This separation gives us greater control over what the user experiences.

Here’s a diagram to help visualize the different kinds of messages and how what’s displayed to the user differs from what’s happening behind the scenes.

The takeaway here is that there are essentially two versions of the same conversation being represented and the Elixir LangChain library helps manage the one with ChatGPT.

Okay, so we know there’s two representations of the conversation, but how do we make ChatGPT act like a fitness trainer?

Prompt Engineering

The term “prompt engineering” has been thrown around so much it’s almost become a joke, but really, that’s a lot of what we’re doing. We’re describing how the LLM should behave. We put boundaries on it, constrain it to a specific context, assign it goals, and provide examples of how we want it to respond.

Prompt Engineering gives us more consistent behavior. If we don’t do it, then our personal trainer might lean British one time, referencing kilogram weights, then the next time it sounds American and use pounds. Or the AI’s personality feels flat, formal, and not encouraging. Or it doesn’t follow-up with our previous workouts.

Prompt Engineering really is just a different kind of coding. It’s providing context, limits, instructions, conditions, goals and examples of desired outputs.

While building this, my AI trainer kept formatting my weekly workout schedule in a seemingly random format each time. So I specified the format to use and where different pieces of data should go so I always get a consistent result. Now it’s consistently how I want it!

The process of prompt engineering takes time and quite a few experiments. We may run it 10 times and it’s on the 11th that we get an unexpected behavior we like and want more of (which we can specify) or we’ll see it do something we want to avoid.

Honestly, that’s part of the fun!

It’s not perfect… but it’s still worth it.

My Proof of Concept Fitness Agent isn’t perfect. The more I use it over time, the more ways I see it can be improved.

But here’s the point

In a short amount of time, I created something that has a meaningful, positive impact in my daily life. I created a tool that I interact with in a natural, conversational way and it helps me stay focused on my goals, and even tracks my progress. And I have fun doing it too!

It make me wonder what else I might want to create… 🤔

Customize your personal AI fitness trainer!

In the demo project, you’ll find a big blob of text that’s submitted as the “system” message for the LLM. It defines how I wanted my AI trainer to behave. This is where you can have fun! Customize your own trainer to be what you want. Here are some suggestions to get you thinking:

Would you prefer a male or female trainer?
Do you want them to be encouraging and positive or are you more motivated by a drill sergeant personality?
Do you want them to focus on strength training, endurance, flexibility, or something else?
Do you want them to use a specific training style? Powerlifting? Bodybuilding for appearance? Crossfit?
Do you want them to be American, British or something else?
Do you want them to speak really casual and use slang? You need to instruct them to.
Do you want them to assume pounds or kilograms? Assigning a nationality brings this kind of context.

Oh the fun you can have! And, in the end, you are the winner. You get an AI Personal Fitness Trainer that works the way you want, and if you use it, you might even reach some personal fitness goals! Now, wouldn’t that be cool?

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

LLaMa on a CPUs

2023-10-11T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Large Language Model (LLM) based AI models are all the rage and running them typically requires a pretty beefy GPU or paying for Open AI API access. So I wondered: Can I run a LLM Model on a normal Fly Machine?

Locally on my M2 Mac, I’ve been able to run the incredible llama.cpp project with pretty solid performance. Let’s see how far we get on Fly machines!

llama.cpp

For those unfamiliar, llama.cpp is a C++ implementation of Facebook’s LLaMA model, optimized for executing on normal CPU’s. It will use whatever matrix processing code your CPU makes available and will use all of your ram. They also use quantization tricks to make it require less memory. I’m not an expert, but understanding is they do some math magic and instead of using Float32 they use Float8, and it somehow just works.

Mistal 7b Instruct

The model we’re going to use is the Mistral 7b Instruct model. Luckily for us here is one that’s been pre-quantized just for llama.cpp. Specifically the Q5KM model variant because it requires just shy of 8gb of ram, and will fit into a normal sized CPU machine on Fly.

Setup Fly

On Fly.io a Docker image approximately maxes out around 8gb, so we won’t be able to bake in our model to the Dockerfile, but we can have llama.cpp ready to go, and this is a Elixir Blog, so we’ll also include our dependencies for that.

    ARG ELIXIR_VERSION=1.15.6
ARG OTP_VERSION=26.1.1
ARG UBUNTU_VERSION=jammy-20230126

FROM "hexpm/elixir:${ELIXIR_VERSION}-erlang-${OTP_VERSION}-ubuntu-${UBUNTU_VERSION}"

RUN apt-get update -y && apt-get install -y build-essential git libstdc++6 openssl libncurses5 locales \
    && apt-get clean && rm -f /var/lib/apt/lists/*_*

WORKDIR "/app"

COPY app.exs /app
COPY port_wrapper.sh /app

# Set the locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8
ENV MIX_HOME="/data/mix"
ENV MIX_INSTALL_DIR="/data/mix"

RUN mix local.hex --force && \
    mix local.rebar --force

# Setup LLaMa.cpp
RUN mkdir "/ggml"
run git clone --depth=1 https://github.com/ggerganov/llama.cpp.git /ggml
WORKDIR "/ggml"
RUN make

WORKDIR "/app"

# Appended by flyctl
ENV ECTO_IPV6 true
ENV ERL_AFLAGS "-proto_dist inet6_tcp"
ENV SHELL=/bin/bash
ENV GGML_PATH="/ggml/server"
ENV MODEL_DIR="/data/models"
ENV WRAPPER_PATH="/app/port_wrapper.sh"

CMD ["elixir", "app.exs"]

  

This is a fairly normal Elixir Dockerfile with the addition of cloning LLAMA.cpp and building it with make. We are also going to need to add a volume, so let’s add that section to a Fly.toml

app = "ggml-example"
primary_region = "ord"
swap_size_mb = 1024

[mounts]
  source = "data"
  destination = "/data"

fly apps create ggml-example
fly vol create -s 10 -r oad data -y
fly scale

This should make our app, create and attach a volume and now we’re finally ready to write some code:

    Mix.install([
  {:req, "~> 0.4.3"}
])

url =
  "https://huggingface.co/TheBloke/Mistral-7B-Instruct-v0.1-GGUF/resolve/main/mistral-7b-instruct-v0.1.Q5_K_M.gguf"

dir = System.get_env("MODEL_DIR") || "./models"
path = "mistrial-instruct.gguf"
full_path = Path.join([dir, path])

File.mkdir_p!(dir)

unless File.exists?(full_path) do
  Req.get!(url, into: File.stream!(full_path))
else
  IO.puts("#{full_path} already downloaded")
end

  

This code will set up our project with my favorite dep Req does a basic check that we’ve downloaded the file, if not we download it to our volume.

And finally, we add a step to execute the code at the bottom:

    ggml_dir = System.get_env("GGML_PATH")
ggml_exec = Path.expand(Path.join([ggml_dir, "main"]))
model = Path.expand(Path.join([System.get_env("MODEL_DIR"), "mistrial-instruct.gguf"]))

prompt = "Tell me a story!"

System.cmd([
  ggml_exec,
  "-m", full_path,
  "-c", 4096,
  "--temp", 0.7,
  "--repeat_penalty", 1.1,
  "-n", -1,
  "-p", "\"<s>[INST]#{prompt} [/INST]\""
  ])
|> IO.puts()

  

Here we basically setup the command to call main with the model, and prompt as per the instructions on hugging face

Finally, let’s deploy and watch the magic!

fly deploy --vm-cpu-kind=shared --vm-cpus=4 --vm-memory=8192

This command is deploying on the shared-4x-8gb Machine, which is essentially the smallest machine you can run on. I typically got a reply in ~5 seconds, and it’s better if you give it a bigger machine.

If you’d like to play around with a deployed version with a little LiveView UI here you go! https://fly-ggml.fly.dev/

What’s next?

Llama.cpp comes with a server mode, so what I did is started its server and called it using Req, streaming the results back. I see this to be useful for if you need to batch process something with a LLM and don’t want to spend a ton of money.

While this won’t be as fast or performant as a GPU or TPU, it’s a great way to get started with LLMs and see what they can do. I’m excited to see what people do with this, and I’m sure there are other models that will work well with this setup.

Please reach out if you find something neat!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Announcing LangChain for Elixir

2023-09-27T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

I created an Elixir LangChain library called “langchain” on Hex.pm. I didn’t invent the idea of LangChain. In fact, it was originally created in Python and JS/TS. I wanted something similar to exist for Elixir and Phoenix applications.

What is LangChain?

Wikipedia sums it up well:

LangChain is a framework designed to simplify the creation of applications using large language models (LLMs).

In short, the Elixir LangChain framework:

makes it easier for an Elixir application to use, leverage, or integrate with an LLM
abstracts away differences between various LLMs
removes boilerplate
provides tools to automate common and even complex tasks

Which LLMs does it support?

With the initial release (v0.1.0), only ChatGPT is supported. The library is built in a way that adding more LLMs can hopefully be done with minimal or no changes needed to an application.

Expect more LLMs to be supported in the future. Here’s the short list:

Meta’s Llama 2
Google’s Bard

Impatient? Get involved! 🙂

Who is this for?

If you want to use Elixir to play with ChatGPT or other LLMs, then this is for you!

What kinds of things can I do with it?

Out of the box you can:

have a conversation with an LLM (more on this up next)
stream the results back
use templates to build prompts (ie LangChain.PromptTemplate)
build complex, multi-level prompts (ie LangChain.PromptTemplate.format_composed/3)
use an LLM to extract structured data from text (ie LangChain.Chains.DataExtractionChain)
integrate the LLM into your Elixir application using a LangChain.Function
define your own tools to expose to the LLM like the included LangChain.Tools.Calculator

There’s more to talk about with each of these which we can hopefully cover that in future posts.

The point is, this is a solid foundation that can already do quite a bit. If you compare this to the more established JS LangChain, you’ll realize there’s a lot more this library could do. But hey, this is just getting started!

I should also make it clear that I’m not trying to match the JS LangChain library feature for feature. I’d like to see some things they don’t do and they include things I think are a terrible idea. 😄

Getting Started

Here’s the basic “hello world” example. After making your OpenAI API key available to the library, the code looks like this:

    alias LangChain.Chains.LLMChain
alias LangChain.ChatModels.ChatOpenAI
alias LangChain.Message

{:ok, _updated_chain, response} =
  %{llm: ChatOpenAI.new!(%{model: "gpt-4"})}
  |> LLMChain.new!()
  |> LLMChain.add_message(Message.new_user!("Hello world"))
  |> LLMChain.run()

response.content
#=> "Hello! How can I assist you today?"

  

Here’s what’s happening in the above example:

We set up the LLM we want to use. In this case, it’s the ChatGPT model from OpenAI. We specify that we want to use the “gpt-4” version of the model.
We start a new chain. A “chain” is the heart of the library. It connects a number of different things together and provides a lot of conveniences.
Next we create and add a new user message to the chain. This is our request.
Finally we run the chain, which sends the conversation to ChatGPT. ChatGPT generates a response which receive back in a new response message.

If we held on to the updated_chain returned from the run function, we’d have a new LLMChain with a new message added from the assistant.

Conversational LLMs are built on the idea of “messages”. A chain or sequence of messages makes a “conversation”. We use a LangChain.Message struct to model this for us.

Stateless Conversations?

Normal HTTP requests are stateless, right? We rebuild the state of the world with every request. Conversational LLMs are the same way! They are stateless! 🤯

If you’ve ever played with ChatGPT, then it’s really fun when you realize you can write something that refers to a previous part of the conversation and magically it works and knows what you’re talking about!

The reason this works is because every time we add a new message or want to continue the conversation and say something else, we pass along the entire conversation up to this point with our addition tacked on at the end.

Here’s a visual example of the different message types in a typical conversation.

Because of this stateless nature, if we store a conversation and pull it up a week later, we can add a message, submit it, and the LLM treats it like there was no pause or delay!

The LLMChain module helps manage the conversation for us. And yes, it makes it easy to stream the results back and apply them to the conversation too. 😁

Where to Start?

For those who have used Livebook before, check out the project’s Livebook notebooks with functional examples.

Otherwise, the project’s installation section should get people going.

The online documentation also includes numerous examples.

Until next time, happy hacking! 👋

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

DIY Lambda

2023-09-21T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

At Fly.io, we make it super easy to spin up new apps and scale them to Machines distributed around the globe. Also, Elixir is uniquely suited to take advantage of this distributed global network. In this post I am going to show you how to use both to:

Start a Machine anywhere on the globe.
Execute whatever code you want on it.
Communicate with it as if it was running locally.

All of this using only the tools provided by Elixir, the BEAM Virtual Machine and one external dependency.

Distribution

Assuming you’ve already set up clustering with your Fly.io app, and deploy your application globally, using the following command:

    fly scale count 5 --region="ord,cdg,nrt,jnb,scl"

  

This will scale your app to 5 Machines, in Chicago, Paris, Tokyo, Johannesburg, and Santiago, covering the major continents with NA, EU, ASIA, AFRICA, and SA.

Once the deployment is complete, you now have a fully connected and globally distributed Elixir cluster. Without doing anything else, you now can communicate with any node, and any process on any node, as if it was running locally. If your application does a PubSub.subscribe(:my_app, "events") in ORD and an event is broadcast PubSub.broadcast(:my_app, "events", "big event!") in JNB it will work transparently.

Further, Fly.io handles network encryption via WireGuard, so your machines know how to communicate and securely by default.

More Machines!

Let’s say we have a very big one-off task we have to complete for our customers in ORD, but we don’t want to slow down our app. Ideally, this task runs on its own machine and reports the results back to your web server.

At Fly.io, you can create, update, and delete machines via our JSON API so let’s do that! Let’s add my favorite Library Req.

    {:req, "~> 0.4.3"}

  

Let’s add a deployment token to our environment secrets. This will give us the access we need to create machines via API:

    fly secrets set FLY_API_TOKEN=$(flyctl tokens create deploy)

  

Then if we want to create a machine, it’s as simple as one HTTP Request:

    app_name = System.fetch_env!("FLY_APP_NAME")
token = System.fetch_env!("FLY_API_TOKEN")
image = System.fetch_env!("FLY_IMAGE_REF")
region = System.fetch_env!("FLY_REGION")
body = Req.post!("https://api.machines.dev/v1/apps/#{app_name}/machines",
  auth: {:bearer, token },
  json: %{
    name: "#{app_name}-async-#{Enum.random(0..100000)}",
    region: region,
    config: %{
      image: System.fetch_env!("FLY_IMAGE_REF"),
      size: "shared", # Choose your size
      auto_destroy: true,
      env: %{
        PHX_SERVER: "false", # If running phoenix
        PARENT_NODE: to_string(node())
      }
    }
  }
).body

{node_name, flyid} =
  case body do
    %{"private_ip" => ip, "id" => id} -> {:"#{app_name}@#{ip}", id}
    error ->
      Logger.error(error)
  end

  

And that’s it, we started a machine! Breaking this code down a little bit:

    app_name = System.fetch_env!("FLY_APP_NAME")
image = System.fetch_env!("FLY_IMAGE_REF")
region = System.fetch_env!("FLY_REGION")

  

First, we are pulling environment variables for the app name and image that’s available on every Fly Machine.

The next part is doing the HTTP Request to the Machines create API. The key bits are here:

      name: "#{app_name}-async-#{Enum.random(0..100000)}",
  region: region,
  config: %{
    image: image,
    size: "performance-2x", # Choose your size
    auto_destroy: true,
    env: %{
      PHX_SERVER: "false", # If running phoenix, skip it
      PARENT_NODE: to_string(node()) # Helper if you need it
    }
  }

  

We choose the basic configuration of the app. The key bit is actually the image setting, we need to make sure the code running on the remote Machine is the same as our current Machine. By manually selecting the image we save ourselves from running this during a deployment when the Machine image might change on us.

Next up let’s connect and execute some code on it:

    # Give the process ~30 seconds to try and connect
Enum.take_while(0..30, fn _ ->
  Process.sleep(1000)
  not Node.connect(node_name)
end)

true = Node.alive?(node_name)

big_data = 0..100000000000000000
map_reduced = :erpc.call(node_name, fn ->
  result = big_data |> MyApp.BigData.map_reducer()

  PubSub.broadcast(:my_app, "events", "big data done!")
  result
end, :infinity)

  

Machines should start quickly, but you know this is a distributed system, and it might take longer than we’d hope, so let’s simply try connecting with a sleep till it connects:

    # Give the process ~30 seconds to try and connect
Enum.take_while(0..30, fn _ ->
  Process.sleep(1000)
  not Node.connect(node_name)
end)

# Final check to make sure we didn't fail completely
true = Node.alive?(node_name)

  

Finally, we simply call our anonymous function on the remote node:

    big_data = 0..100000000000000000
map_reduced = :erpc.call(node_name, fn ->
  result = big_data |> MyApp.BigData.map_reducer()

  PubSub.broadcast(:my_app, "events", "big data done!")
  result
end, :infinity)

  

You may have noticed we didn’t do anything at all to serialize our data to the remote node. It will just serialize the data for us, send it along, and return the result transparently. This is the magic of the BEAM, this isn’t limited to simple values either, send files, do IO, anything, the BEAM will figure it out.

Since we’re working with our existing code, we can just use it like any normal code, no special syntax or libraries. Whatever code we deploy to our normal app image we can execute and use normally. For example the PubSub we used above should just work for any node listening for events on :my_app.

When finished, use the fly_id to destroy the Machine via API:

    Req.delete!("https://api.machines.dev/v1/apps/#{app_name}/machines/#{machine_id}",        auth: {:bearer, token })

  

Alternatively you can simply tell the Machine to shutdown like so:

    :erpc.call(node_name, fn ->
  System.stop()
end)

  

Because of auto_destroy: true in our creation config, when the system stops the machine will destroy it’s self.

Recap

We successfully started, executed our big data code, and stopped a machine all from Elixir with only an HTTP Client. In the BEAM world we tend to take this for granted but let’s step back and list out the pieces we’d have to build or include in our running system if we did this in any other environment!

Direct Port:

Some sort of HTTP/RPC server.
Choose how to serialize AND deserialize our data on both ends.
Tooling to handle errors. In our case, any error from :erpc call will propagate to the calling Machine automagically.
Tooling to start/stop Machines:
- Deploy our code to them.
- Start the code.
- Make sure it can be discovered and communicated to securely.

Lambda/Worker Style:

Redis/Kafka/RabbitMQ/SQS aka message broker
Setup/Deploy/Maintain a worker
- Lambda, which is not written in Elixir and billed opaquely.
- Sidekiq/Oban, which runs continuously and can’t be scaled up/down
Ability to monitor.

We get all of that for free thanks to the BEAM and Fly.io working together. Now obviously putting this all into production will require making engineering decisions such as:

Is this work so important it can’t fail?
Can I handle some downtime if the network/host fails?
How do I handle errors?
Durable data?

Fortunately, the BEAM has been working on these problems since the 80s and is chock-full of tools and educational content to help you make these decisions.

Conclusion

We created a one-off Machine, connected to it, executed a function, and got the result. Our Machine runs using our code in our deployment environment, no special third party code is required. Our Machine runs for only as long as it is needed and no longer keeping our spend in check. We did it with none of the usual code we’d expect to need in other languages or environments, this is incredible!

What’s next?

Soon we plan to release a library that makes all of this much easier, so if you want to run something on a new Machine it’s as simple as Task.async, keep an eye out for it! Until then, we have set up the building blocks to do it yourself.

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Customizing Phoenix Generators

2023-09-11T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

During my interview with Victor Björklund, I learned how easily the Phoenix generators can be customized for our projects. I was shocked I that I never knew about it and I don’t recall ever seeing it documented. This post is to document this invisible feature that is super easy to use and can be really powerful for teams with either new or mature projects.

Problem

You have an established project and over time, patterns and styles were developed for LiveViews, Controllers, etc. Now, every time you add a new Context, LiveView, Controller or some other standard thing, rather than use the built-in generators, you’ve been copying an existing file and changing it to be what you want.

The process of copying and tweaking files this way is tedious and error prone. We frequently end up with left-overs in our resulting code that points back to where it was copied from.

Is there a way to make creating new standard parts of our application smoother? Can we customize an existing Phoenix generator to make it match our application and our patterns?

Solution

Yes, it’s actually easy to do!

First, let’s be clear about which Phoenix generators we’re talking about.

For your Phoenix project, type mix phx.gen to list the available Phoenix generators. This is what mine looks like:

mix phx.gen

mix phx.gen.auth     # Generates authentication logic for a resource
mix phx.gen.cert     # Generates a self-signed certificate for HTTPS testing
mix phx.gen.channel  # Generates a Phoenix channel
mix phx.gen.context  # Generates a context with functions around an Ecto schema
mix phx.gen.embedded # Generates an embedded Ecto schema file
mix phx.gen.html     # Generates context and controller for an HTML resource
mix phx.gen.json     # Generates context and controller for a JSON resource
mix phx.gen.live     # Generates LiveView, templates, and context for a resource
mix phx.gen.notifier # Generates a notifier that delivers emails by default
mix phx.gen.presence # Generates a Presence tracker
mix phx.gen.release  # Generates release files and optional Dockerfile for release-based deployments
mix phx.gen.schema   # Generates an Ecto schema and migration file
mix phx.gen.secret   # Generates a secret
mix phx.gen.socket   # Generates a Phoenix socket handler

Every one of these can easily be customized to generate something more appropriate for our project! Let’s see how!

Choosing a Generator

To get started, let’s pick a generator to customize. It makes sense to choose one that we’ll continue to use as our project evolves. For this example, we’ll use the LiveView generator for a resource. In our case, it’s the mix phx.gen.live task.

With our generator selected, let’s go!

Step 1: Locate the current template

Luckily, we don’t have to start from scratch! Here’s how we can find the current generator that is used for our project.

Our project’s mix tasks come from our mix dependencies. Specifically, this mix task comes from our Phoenix dependency and lives in deps/phoenix. The generator templates are found in deps/phoenix/priv/templates.

Alternatively, we can find the templates online. Here’s a link to the templates on the Phoenix Framework’s main branch. We can use the Github tags to find specific versions of the templates as well.

Step 2: Copy the files we want to customize

Now that we’ve found the existing files to start from, we can copy the templates into our project.

This brings up the next question, “Where should these go?”

To answer this, let’s explore a bit about how the mix task templates work.

How does this work?

The template files are in the Phoenix project’s priv/templates folder. It turns out that the Phoenix generators first look in our project’s priv/templates folder for the template files. When the templates aren’t found in our project, and they usually aren’t, it falls back to use the Phoenix project’s templates.

Take-away

If we put one or more customized template files in priv/templates/MIX-TASK-NAME/ then the mix task will look for the file in our project first. Any files we don’t include there will fall back to the Phoenix generator.

The phx.gen.live directory name is used because that’s the mix task we’re running and where the original templates are located.

This example is using the phx.gen.live generator. In this case, our files will end up in our project’s priv/templates/phx.gen.live/ directory.

Let’s create that directory in our project.

Step 3: Customize the generated LiveView `index.ex` file

Let’s walk through what we need to do:

copy the file(s) to customize into this directory. For this example, we’ll copy the index.ex file which generates the LiveView’s code for the index page.
make a change to index.ex. For our example, well just add the comment # CHECK IT OUT! Custom change right here!!!.

Here’s the first part of our modified priv/templates/phx.gen.live/index.ex template file. Notice the “CHECK IT OUT!” comment was added.

    defmodule <%= inspect context.web_module %>.<%= inspect Module.concat(schema.web_namespace, schema.alias) %>Live.Index do
  use <%= inspect context.web_module %>, :live_view

  alias <%= inspect context.module %>
  alias <%= inspect schema.module %>

  # CHECK IT OUT! Custom change right here!!!

  @impl true
  def mount(_params, _session, socket) do
    {:ok, stream(socket, :<%= schema.collection %>, <%= inspect context.alias %>.list_<%= schema.plural %>())}
  end

  

Let’s run it and see what happens.

Step 4: Run the generator

Let’s run the mix task using the following command:

To see all the options available for executing this generator, use the help command like this: mix help phx.gen.live

mix phx.gen.live Foos Foo foos name:string age:integer

The mix task generated a bunch of files. Let’s see the generated index.ex file:

    defmodule Web.FooLive.Index do
  use Web, :live_view

  alias Web.Foos
  alias Web.Foos.Foo

  # CHECK IT OUT! Custom change right here!!!

  @impl true
  def mount(_params, _session, socket) do
    {:ok, stream(socket, :foos, Foos.list_foos())}
  end

  

Notice our change (the comment) is included in the newly generated module. The other thing to note is that all the other files were generated without customization.

Perfect! It worked! We customized a Phoenix generator template file and it was used by our project.

Discussion

It turns out that the Phoenix team created an excellent escape hatch for the Phoenix generators!

What I love about this is approach is our customizations are checked-in with the project. Our team members get access to the generators too. As we define new patterns for our project, we now have a good place to put them!

We saw here that we can customize just one, or many of the template files for a generator. It’s totally open for us to decide.

Keeping templates out of our deploy

By default, any templates in our priv/templates directory will be included in our deploy. There’s nothing inherently dangerous about that, but still, why include dev-focused files in our production build? Luckily, it’s easy to prevent those files from being included.

Assuming we’re using a Dockerfile to deploy, then adding the following line to our .dockerignore file excludes any and all templates from the build.

priv/templates/

Nice and easy.

What can we do with this?

With this, we can customize how new LiveViews are generated. We can tailor them for our application, to use our components, our styles, our helper modules, our markup, all the things we used to copying over but often ended up copying too much.

Is it just LiveViews? No! We could customize Contexts with all the functions you end up adding anyway. We could customize migrations, schemas, controllers, or whatever our application uses frequently.

If you’re happy with your established patterns, you can copy them to a brand new project and use it from the beginning.

Cool trick!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Streaming Uploads with LiveView

2023-08-31T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Time to face reality. Sending files over the internet via a web browser is not a solved problem. There has been so many bytes spilled on the varied ways of accepting, uploading, streaming, verifying, storing, backing up, and sharing files from a user. It’s 2023, and I realize I’m not getting a flying car, but you’d hope it was as simple as:

    <input type="file">

  

Unfortunately, everyone knows that’s only the very tip of the iceberg, and you can quickly ‘titanic’ the whole project by not taking this one seriously.

That’s not to say there aren’t many third-party solution to make this easier; by all means please tweet at me about them to increase my engagement. Often the very best solution is “submit the form to s3 and forget about it”, this is totally not annoying to configure while attempting to keep spend low. If, you are reading this as a “how to do uploads” post, then stop here and consider doing just that.

The second it’s time to step outside the happy path and do anything more complex, we end up with abominations. Building with state machines, job queues, SQS, Lambdas, ImageMagick, FFmpeg, python, CSV’s, excel and the list goes on. Often including a handful of third party services because we’ve frankly lost our nerve.

Again, once you’ve built the abomination, you can often be “done” with it and move on. So if that’s you, please stop reading and tell me about it on social media.

But what if there was a better way?

Enter LiveView Uploads

With LiveView, one of the very early and ambitious efforts for the team was to try and get this one right. Since the entire page lifecycle for a LiveView is over a WebSocket we had to think really hard and make some engineering trade offs. Today, if you want to do a basic upload to the current file system LiveView has your back. Call allow_upload\3 in your mount callback

    allow_upload(socket, :avatar, accept: ~w(.jpg .jpeg))

  

in this case only allowing a single file with extensions .jpg or .jpeg and then adding live_file_input\1 to your form

    <.live_file_input upload={@uploads.avatar} />

  

and finally in your form submit function you need to consume_uploaded_entries/3

    consume_uploaded_entries(socket, :avatar, fn %{path: path}, _entry ->
  dest = Path.join([:code.priv_dir(:my_app), "static", "uploads", Path.basename(path)])
  File.cp!(path, dest) {:ok, ~p"/uploads/{Path.basename(dest)}"}
end)

  

It’s all kinda hand wavy and magic, because well it is! We’re hiding an incredible amount of complexity here, and in the end you get a complete file path that you can move to wherever is best for you! AND when the webpage closes, the “temporary” file get’s cleaned up! We even get real time upload progress, drag and drop and the ability to handle multiple uploads at once, for free.

Now the grizzled file upload veteran is already shaking their head to point out some serious downsides with this as a base method:

What if the file is extremely large?
What if we want to handle the file as a stream of bytes and process in real time?
What if I want to send this file to many places? Now I need to copy it?
What if that file is not cropped correctly?
It’s still going to want to go to s3…
and so on

Prior to just a month ago, the answer for all of these questions was to upload directly to s3 and figure the rest out later. Which, as alluded to above, is a totally rock solid solution.

UploadWriter: Chunk-by-Chunk Processing & multi-destination File Uploads

Enter the LiveView.UploadWriter which, when configured in allow_upload\3 under the option :writer, can handle file uploads in chunks, effectively streaming the file to the backend allowing you full control over how the file is consumed. In fact, the default writer is the UploadTmpFileWriter which generates a random tmp file opens it for writing and writes the chunks as they arrive, closing the file at the end.

Let’s make our own UploadWriter that sends the file to two places!

    defmodule DoubleWriter do
  @behaviour Phoenix.LiveView.UploadWriter

  @impl true
  def init(_opts) do
    file_name = Utils.random_filename(".jpg")
    with {:ok, path} <- Plug.Upload.random_file("local_file"),
         {:ok, file} <- File.open(path, [:binary, :write]),
         s3_op <- ExAws.S3.initialize_multipart_upload("bucket", file_name)
         do
      {:ok, %{path: path, key: file_name, chunk: 1, s3_op: s3_op, s3_config: ExAws.Config.new(s3_op.service}}
    end
  end

  @impl true
  def meta(state) do
    %{local_path: state.path, key: state.key}
  end

  @impl true
  def write_chunk(data, state) do
    case IO.binwrite(state.file, data) do
      :ok ->
          part = ExAws.S3.Upload.upload_chunk!({data, state.chunk}, state.s3_op, state.s3_config)
          {:ok, %{state | chunk: state.chunk+1, parts: [part | state.parts]}}
      {:error, reason} -> {:error, reason, state}
    end
  end

  @impl true
  def close(state, _reason) do
    case {File.close(state.file), ExAws.S3.Upload.complete(state.parts, state.s3_op, s3_config)} do
      {:ok, {:ok, _}} ->
          {:ok, state}
      {{:error, reason}, _} -> {:error, reason}
      {_,{:error, reason}} -> {:error, reason}
    end
  end
end

  

Thanks to s3 being a bear there is maybe too much going on in here but lets walk through it:

def init(opts) do Initializes the Writer, in our case we initialize an Empty Tmp File, and start an S3 Multipart Upload, and that becomes our initial state.
def meta(state) do Return the metadata from state that we might use on the front end
def write_chunk(data, state) do Gives you the chunk to write to the file and s3, we also store some state for cleanup.
def close(state, _reason) do This is called when the file is done uploading, or errors for any reasons. In our case, we don’t handle the error, we close the File and complete the s3 upload. Returning any errors that might pop up.

To use DoubleWriter, update our allow_upload from earlier:

    allow_upload(:avatar,
  accept: ~w(.jpg .jpeg),
  writer: fn _name, _entry, _socket -> {DoubleWriter, []} end )

  

We could have passed the name/entry to our writer to give it a more friendly name or do more details checks in our init.

Taking a step back what this gets us is fully Streamed, chunk by chunk, uploads to wherever we want them. In our case it would be better to upload the file directly to s3 from the client and save on ingress/egress costs. There are some use cases where you might prefer this for security and compliance reasons, for example maybe you don’t wanna pre-sign an upload and send it to the client.

Could also imagine processing a CSV as it came in building up lines from chunks as they arrived, creating UX around the header line, and then filling in a table in real time. Say we know a line of a CSV is 100 bytes, we could align our chunks to be 100 bytes and process them line by line as they arrive.

Or real time thumbnail generation.

Or scanning a file for a specific value and then cancel the upload once you have the data you need. The list goes on

Wrap up

While this is no flying car or as clean as a single file input, it is cleaner than building up all of this infrastructure from scratch. Using these primitives we can process files of nearly any size; receiving them, processing them and sending them as quickly as they arrive.

And please share with the Phoenix team what you build with this!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Star-Crossed LiveView Processes

2023-08-29T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Ever wanted to create something like a ChatGPT interface that asynchronously streams in a response? I have! Assuming we’re building it with LiveView, then we want to run that streaming conversation in a separate process because we don’t want to block our LiveView from being highly responsive to user input. In this post, we’ll cover how to build that type of async code in our LiveView using the awesome building blocks available to us in Elixir. We’ll also have some fun learning a bit more about Elixir’s concurrency primitives along the way!

Dramatic Version: 🎭 This is a story of two star-crossed ~~lovers~~ processes. When linked, their fates are intertwined and a tragedy spreads it’s destruction. Can the fates be changed? Perhaps… but at what cost? Actually, it’s really easy. 😆

What we’re solving

Elixir is awesome at concurrency. Processes, links, and monitors form a powerful foundation. Built on top of that, we have Supervisors, Tasks and even LiveView. Sometimes, the challenge can be knowing how to use the different tools available to us to build what we want.

In this case, there isn’t a ready-made solution already built for us called something like a “AsyncPatternGoodForChatGPTStyleUI”. Fortunately, it isn’t hard to build what we want. In fact, once we see how easy it is, it’ll hopefully be clear why we don’t need a pre-built library for it.

Let’s visualize what we’re building. It’s a LiveView that launches a Task, and we’re more interested in the messages sent back while that Task is running than the final result of the Task. Our focus on the messages during the process changes how we design our solution. If you are more focused on the final result of a Task, then check out this excellent post.

Before we jump into the code, let’s define a bit more of what we want the code to do.

Goals

Here we’ll outline how we want the application to behave, particularly around the async process and our LiveView.

The LiveView process remains unblocked. Our blocking external calls should happen in a separate process. Let’s keep the UI buttery smooth.
When the LiveView process goes away, the other process should too. Because the worker process only exists to fetch and provide data to the LiveView, we want the async process to stop if the user closes the page or navigates away.
When the async process crashes, it should NOT kill our LiveView. We expect the async process talking to an external API will fail sometimes. We do not want that to crash the UI for the user.
Ability to cancel a running worker process. We want the ability to cancel a running async worker from the LiveView.
We care about the side-effects, not the final result. Thinking back to ChatGPT as an example, the stream of text coming in is the side-effect we want. We could wait for the full text to be collected and then returned as the final “here it all is!” result, but that’s not actually what we want. The user can start reading the streamed in result long before the full text is available. These small chunks of data can be sent over as they are received and this is the side-effect we’re talking about.

All right! That may seem like a hefty set of requirements, but it’ll give us the user experience we want and is actually easier to do than you may suspect.

Here is a visual example of the behavior we want.

What Building Blocks to Use?

Well, we know we want a separate process to do the work of talking to an external API. We could use spawn, a GenServer, or a Task. Actually, both GenServer and Task are abstractions on top of spawn. For this solution, we’ll use a Task. Why? We only need the additional process to run when there’s an API request to make. That’s actually a really good fit for a Task.

Great, so we know we want a Task, but there are a number of different ways to run one. Ever gotten confused about how they differ? Yup. Me too.

The one we want is Task.start_link/1. Why? This one links the spawned task to our LiveView process. Let’s look at that next.

Thinking about Linking

Processes can be easily linked manually using Process.link/1 or when starting a new processes using spawn_link/1. In our situation, we want a Task and when we call Task.start_link/1, it makes it easy to link the Task process to our LiveView.

When two processes are linked together and one process dies, the linked partner dies as well. It’s Romeo and Juliet on the BEAM. 💕😵 So sad.

A visual story of our linked star-crossed processes:

When the Juliet process dies…

…a signal is sent to the Romeo process, killing it as well.

It’s important to note that this linking process goes both ways. If the Romeo process died first, a signal would be sent to the Juliet process killing it.

Tip: I find it helpful to think about processes as people. This makes the interactions easier to understand and feels more natural.

Back to our Problem

When we start a Task using Task.start_link/1, it creates the process and links it to our LiveView process for us. Neat!

This means when our LiveView closes, any running Task automatically gets killed. But wait… doesn’t that mean that when our task ends, it kills our LiveView? Uh… yes. But hold on! Unlike the famous Shakespearean play, our processes can change their fates.

The signal sent when a process dies is a special system-level message telling the other linked process to exit. Normally, we never see those messages. However, if we use this snippet of code during the setup of our LiveView (like in the mount), then we will see those messages telling our process to exit.

    Process.flag(:trap_exit, true)

  

If we remember that our LiveView is a process, then this elegant solution isn’t far fetched. This is also referred to as trapping exits.

Now, when our linked Task either completes normally or crashes, the LiveView gets the message that it should exit also. It comes in as a handle_info callback and looks like this:

      def handle_info({:EXIT, pid, reason}, socket) do
    IO.puts("Received exit signal for pid #{inspect(pid)} with reason: #{inspect(reason)}")

    {:noreply, socket}
  end

By catching this message and NOT returning a response like {:stop, reason}, we effectively handle the request for our LiveView process to close, but choose to not close. Excellent! That means we’ve got the linking working the way we want.

Let’s recap how the linking will behave:

When our LiveView closes, it kills a running task.
When our Task crashes/dies/finishes, it notifies our LiveView.

Let’s see what we can do with that.

Trapping and Handling Exits

Here’s a LiveView code sample of what we’ve covered so far. We’ll discuss it after.

      def mount(_params, _session, socket) do
    # Trap exits
    Process.flag(:trap_exit, true)

    socket =
      socket
      # keep track of a running task for the UI
      |> assign(:running_task, nil)

    {:ok, socket}
  end

  def handle_info({:EXIT, pid, reason}, socket) do
    socket =
      if pid == socket.assigns.running_task do
        # the closed PID was our task, remove it
        assign(socket, :running_task, nil)
      else
        socket
      end

    {:noreply, socket}
  end

  

In the LiveView’s mount callback, we setup to trap exits. It would be great if our UI displayed when a task was running and even let us cancel it. To do that, we can track our :running_task as a pid (Process ID).

In our handle_info callback, we keep an exiting Task from killing the LiveView process, but we can also clear our reference to the running Task! An argument in the message is the pid of the process that exited.

This makes it really easy to keep our :running_task value up-to-date without us having to manually track what’s happening with it.

Now we’re ready to create our Task to do some work!

Starting the Task

In this article we’re more concerned with how the LiveView and Task processes interact and we’re not focusing on the actual work the Task is doing.

When the user clicks the “Start” button, the handle_event clears the messages and starts the Task.

      def handle_event("start", _params, socket) do
    socket =
      socket
      |> assign(:messages, [])
      |> start_test_task()

    {:noreply, socket}
  end

  

In our start_test_task function, we use Task.start_link/1 which takes an anonymous function to execute asynchronously.

      def start_test_task(socket) do
    live_view_pid = self()

    {:ok, task_pid} =
      Task.start_link(fn ->
        # the code to run async
        Enum.each(1..5, fn n ->
          Process.sleep(1_000)
          send(live_view_pid, {:task_message, "Async work chunk #{n}"})
        end)

        # function result discarded -- it isn't used
        :ok
      end)

    # returning the socket so it's pipe-friendly
    assign(socket, :running_task, task_pid)
  end

  

Before we start the Task, we call self() and assign the pid of the LiveView process to a variable that can be referenced in our async function. This passes the pid of the LiveView via a closure to our async function.

The “work” being done in our function is looping 5 times, sleeping for 1 second, then sending a message to the LiveView process about the chunk of work we completed.

When the task finishes running the function, it auto-exits.

Concurrency Overview

For those new to Elixir’s actor model concurrency, the code may appear confusing, but it’s actually quite elegant. Here’s a different perspective of what’s happening:

In Elixir, processes are cheap and quick to start. As soon as we start the Task, we get back the new pid (Process ID) and store it in our LiveView’s assigns. The LiveView now knows we’re running an async task and our UI automatically updates to reflect it. Nothing is blocked in our main LiveView process. Yay!

As the Task runs, it does work and sends messages back to our LiveView which we respond to in a handle_info function with the pattern match for the message.

When the Task completes, it auto-exits. Because the processes are linked and we are trapping exits, the LiveView process receives the system :EXIT message and we remove the :running_task pid from the assigns. Our LiveView’s UI automatically updates to remove the “Cancel” button and display the “Start” button again.

What’s amazing to me is how elegantly this approach works and how natural it feels.

More Insight into Task Exits

We’d like some more insight into when our Task exits. After all, we want to make sure it has the behavior we want.

Let’s sprinkle in some IO.puts print messages to see what’s happening.

      def handle_info({:EXIT, pid, reason}, socket) do
    IO.puts("Received exit signal for pid #{inspect(pid)} with reason: #{inspect(reason)}")
    # ...
  end

  def start_test_task(socket) do
    live_view_pid = self()

    {:ok, task_pid} =
      Task.start_link(fn ->
        # the code to run async
        Enum.each(1..5, fn n ->
          Process.sleep(1_000)
          IO.puts "SENDING ASYNC TASK MESSAGE #{n}"
          # raise "TASK RAISED EXCEPTION"
          send(live_view_pid, {:task_message, "Async work chunk #{n}"})
        end)

    # ...
  end


  

We added two IO.puts commands. Now let’s see the results in the console.

Normal completion

This is what a normal Task completion looks like.

SENDING ASYNC TASK MESSAGE 1
SENDING ASYNC TASK MESSAGE 2
SENDING ASYNC TASK MESSAGE 3
SENDING ASYNC TASK MESSAGE 4
SENDING ASYNC TASK MESSAGE 5
Received exit signal for pid #PID<0.6890.0> with reason: :normal

Notice that the exit reason is :normal. This tells us under what circumstances the Task exited. It was done!

If you’re thinking, “Hey, couldn’t we pattern match on the exit reason if we wanted to?” then yes, you’re right! We’ll look a bit more at some other exit reasons in a minute.

Navigate away from the LiveView

One of our goals is that when the user navigates away from the LiveView, a running Task should immediately be cancelled. Let’s see what happens when we navigate away.

SENDING ASYNC TASK MESSAGE 1
SENDING ASYNC TASK MESSAGE 2
SENDING ASYNC TASK MESSAGE 3

We received 3 messages before navigating somewhere else. In the console we see that the Task stopped executing. Excellent!

The Task was notified to close because of the process link. Since the Task isn’t setup to trap exits, we don’t see messages about that happening. What we do see is that the task stopped running as soon as the LiveView process exited. This is exactly what we want here. If the LiveView is gone, the Task should stop immediately and it did.

Task crashes

What happens if the Task crashes? In our Task function, right after our sleep, we’ll add raise "TASK RAISED EXCEPTION" and see what happens.

SENDING ASYNC TASK MESSAGE 1
[error] Task #PID<0.7046.0> started from #PID<0.7035.0> terminating
** (RuntimeError) TASK RAISED EXCEPTION
    (my_app 0.1.0) lib/my_app_web/live/task_test_live/index.ex:81: anonymous fn/2 in MyAppWeb.TaskTestLive.Index.start_test_task/1
    (elixir 1.15.4) lib/enum.ex:989: anonymous fn/3 in Enum.each/2
    (elixir 1.15.4) lib/enum.ex:4379: Enum.reduce_range/5
    (elixir 1.15.4) lib/enum.ex:2514: Enum.each/2
    (my_app 0.1.0) lib/my_app_web/live/task_test_live/index.ex:78: anonymous fn/1 in MyAppWeb.TaskTestLive.Index.start_test_task/1
    (elixir 1.15.4) lib/task/supervised.ex:101: Task.Supervised.invoke_mfa/2
Function: #Function<0.12965136/0 in MyAppWeb.TaskTestLive.Index.start_test_task/1>
    Args: []
Received exit signal for pid #PID<0.7046.0> with reason: {%RuntimeError{message: "TASK RAISED EXCEPTION"}, [{MyAppWeb.TaskTestLive.Index, :"-start_test_task/1-fun-0-", 2, [file: ~c"lib/my_app_web/live/task_test_live/index.ex", line: 81, error_info: %{module: Exception}]}, {Enum, :"-each/2-fun-0-", 3, [file: ~c"lib/enum.ex", line: 989]}, {Enum, :reduce_range, 5, [file: ~c"lib/enum.ex", line: 4379]}, {Enum, :each, 2, [file: ~c"lib/enum.ex", line: 2514]}, {MyAppWeb.TaskTestLive.Index, :"-start_test_task/1-fun-1-", 1, [file: ~c"lib/my_app_web/live/task_test_live/index.ex", line: 78]}, {Task.Supervised, :invoke_mfa, 2, [file: ~c"lib/task/supervised.ex", line: 101]}]}

When our Task crashed, we notice that our LiveView doesn’t crash but still gets notified of the exit. The last message in the console output is for the EXIT. Note that the reason is a tuple with the exception raised. Nice! That could be helpful information, especially if we wanted to pattern match on an error. 😉

In our application, we’re already removing the :running_task from our assigns and our UI updates correctly when the task crashed. No extra work needed there!

Cancelling the Task

One of our goals was to be able to cancel a running Task. How do we do that?

To review, we are tracking the pid of the Task in :running_task. This serves two purposes.

we know when a task is running
we have the pid of the task

Let’s add a “Cancel” button to the UI. The following markup displays a “Start” button when no Task is running and the “Cancel” button when a Task is running.

    <div class="...">
  <.button :if={is_nil(@running_task)} phx-click="start">Start</.button>
  <.button :if={@running_task} phx-click="cancel">Cancel</.button>
</div>

  

Next we’ll handle when the “Cancel” button is clicked. Let’s see how we can do that.

      def handle_event("cancel", _params, socket) do
    task_id = socket.assigns.running_task

    socket =
      if task_id do
        Process.exit(task_id, :kill)
        # display it was cancelled.
        put_flash(socket, :info, "Cancelled")
      else
        socket
      end

    {:noreply, socket}
  end

  

It’s really simple. We send an exit message to the Task using it’s pid. The message we send is :kill to let the system know we want it closed immediately.

Finally, we add a flash message to display that it was cancelled.

Let’s see what happens in the console when we cancel a running Task.

SENDING ASYNC TASK MESSAGE 1
SENDING ASYNC TASK MESSAGE 2
Received exit signal for pid #PID<0.6888.0> with reason: :killed

The Cancel button was clicked and the Task exited. Our LiveView was notified that it exited and the reason was :killed. How cool is that?

The whole “cancel” feature almost feels anti-climactic because it was so easy.

It’s worth pointing out that we don’t have to explicitly handle removing the Task pid from the :running_task in our assigns because even on a cancel, the EXIT message is received and handled in one place. In fact, let’s look more at that next.

Making a Nice, Orderly Exit

I really like how the Task can exit under different circumstances but it all passes through the same handle_info callback. The same handler fires when the Task:

completes successfully
is cancelled
errors and crashes

This makes keeping the UI up-to-date with a Task’s running state very simple and clean.

Also, the reason for the exit is sent along in the message which we can pattern match on to handle in the most appropriate way for our project.

It’s a really powerful feature of the BEAM (the VM runtime) when all we have to do is tell the BEAM that want to be notified when a linked process exits. Love it.

Where We Ended Up

We achieved our goal. We set out to figure out how to architect the building blocks so our LiveView could run an asynchronous Task and receives updates of the progress along the way. The example use-case was a ChatGPT-style flow of messages that builds up a result.

In the process, we created a beautifully working async solution in less than 100 lines of code. Here’s a Gist of the code.

We also learned more about how processes can be linked together and the resulting star-crossed processes’ fates are bound together as well. Then we saw that trapping exits lets us change that fate and enables a simple and elegant solution.

Sometimes the challenge is knowing which of the powerful building blocks are the right tools for our specific situation. Hopefully these building blocks are a bit more approachable now and you’ll see more opportunities for using them in the future too!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView apps. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

BEAM Clustering made easy

2023-08-15T17:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

One of the reasons Fly.io is so great for Elixir and Phoenix is that we have servers in nearly every region on the globe, and they are all connected via built in WireGuard networking. As an Elixir developer, this is an incredible opportunity because it means we get to use Erlang Distribution and clustering with minimal setup, globally.

Distribution?

The history of the BEAM is basically to handle as many concurrent phone calls as possible, and route them to the correct place. So being able to connect to other nodes and send messages transparently from node to node is a necessity.

Without any third party tools you can simply call Node.connect/1 with a host name and ip for example:

    Node.connect(:"name@127.0.0.1")

  

And assuming nothing fails you’ve connected the two BEAM. Send messages, start processes on either machine, the world is your oyster!

Now the sharp readers might be asking…

But if we run on a globally distributed cloud, how do you know which machines to connect to?

Fly.io Private Networking

Fly apps are connected by a mesh network of WireGuard tunnels using IPV6. And in order to set up those tunnels, Fly needs to know, which machines to connect to… which is provided by a Fly .internal addresses. If you do a AAAA DNS query against app_name.internal you will get back a list of nodes to connect to. For example, if we only had one app:

    :inet_res.lookup(~c"my_app.internal", :in, :aaaa)
|> Enum.map(&to_string(:inet.ntoa(&1)))

["fdaa:1:36c9:a7b:198:c4b1:73c6:1"]

Adding a second region is as simple as cloning an existing Fly Machine to another region.

fly machine clone MACHINE_ID --region NEW_REGION

Once setup the above DNS lookup will return two IPV6.

To scale up multiple machines with in a region it’s as simple as

fly scale count 2

Making it Easy

The Phoenix Team has released a new project called dns_cluster which aims to simplify working with clustering in the case where you use DNS for discovery. I highly recommend you read the code to see what it does, it is almost unnecessary to make into a library for a paltry 191 lines of code.

Simply follow the README and add one line to your application.ex and another to runtime.exs and you are off to the races! You can verify that you are connected to another node by calling, Node.list/0 and you’re off.

Wrap Up

There are many ways to connect a bunch of nodes and the go-to library for more complex setup’s is the fantastic libcluster from Paul Schoenfelder (Bitwalker). Using libcluster you can do zookeeper or multicast UDP or K8s, and he even has a DNS strategy as well.

We’ve just started scratching the surface of what’s possible:

Dig into the OTP modules like global or erpc
Try out mnesia the distributed DBMS that ships with OTP
Fly.RPC makes it really easy to execute functions on specific nodes
Try out Phoenix PubSub!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Let's search all of Elixir's Packages!

2023-08-08T15:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

I love ExDocs, I think they are one of the best parts about the Elixir Ecosystem and I frankly cannot shut up about it. My one complaint is that it can sometimes be hard to find exactly what you are looking for using ExDoc’s built in search.

My favorite example is that I know that Ecto.Repo has a function to dump the generated SQL to a string, but if you search inside of Ecto you get nothing. That’s because it’s in a different project called ecto_sql. What if we had one spot we could go to and search all of HexDocs?

In this post we will walk through how I downloaded, cleaned up, indexed and searched all of HexDocs using SQLite FTS5 and LiveView! It turned into a project that matches the real world incredibly well, warts and all! So I won’t be walking through every single step, but check it out on Github and try it out hosted here on Fly.io, let’s begin!

Downloadin’ Hex Docs

When you visit a ExDoc package and try using the search box, what you are using is a fully in-browser search experience provided by Lunrjs. When the ExDoc generated page is first visited it downloads a JavaScript payload that contains the “Search Data”, which is then indexed in the browser via Lunrjs, compressed and stored in the browser SessionStorage. Then there is some more JavaScript that does the autocomplete and rendering of the search fully in the browser.

We’re going to take advantage of that Search Data that comes with every hex package, and it looks like this:

ExDoc generates this for every package and uploads it to HexDocs for you. And because ExDoc has a special place in the world of Elixir, every Elixir project depends on it, including Elixir. So it can’t really depend on any project that’s not Elixir or fully vendored, like Lunr.js.

Which means that when ExDoc creates the above searchData.js file it has to encode the JSON directly. The good news is the “items” all have the same document shape, the, meh news is this filename is different for every deploy.

Garden Variety Extract, Transform and Load Pipeline

This project started as a fun project that scratches an itch for me, but building out an ETL pipeline is as “Real World” as programming gets. Much like the “Real World” this part of the project kept expanding in scope as I went along.

So we don’t get lost in the weed’s here is a little road map to keep us on tract.

E. Use the Hex API get a listing of all the packages

E. Scrape the Search Page of every package that has documentation looking for an script tag that contains search_items and download that file

T. We’re going to strip off all JavaScript in that file till we just have json, and then clean it up.

T. We’re going to parse that JSON using Jason, if that fails use a hand rolled Json parser that works with the cases that don’t work with Jason.

L. Store this into a SQLITE Table

E: Get the data

    Enum.reduce_while(1..500, [], fn page, acc ->
  data = API.list!(%{page: page, sort: "updated_at"})

  current = Enum.count(data)
  if current < 100 do
    {:halt, [data | acc]}
  else
    {:cont, [data | acc]}
  end  
end)
|> Kernel.++(special_packages())
|> List.flatten() 

  

We’re hitting the Hex API page by page and downloading all the data till we get a page less than size 100. I’m also adding in some “special packages” which are just Elixir, Eex, ExUnit, IEX, Logger, and Mix that are hosted on hex but not served via the API. Standard stuff here!

You’ll notice I’m just letting errors happen instead of handling them, this is a “for fun” project but its also practical as it stop’s this process as soon as something fails so I don’t need to re-run it a ton of times.

E: Scrape

    def scrape_json(pkg)
  pkg["docs_html_url"] <> "search.html"
  |> grab_html(search_url)
  |> grab_json(["search_items", "search_data"], url)
end

def grab_html(url) do
  {:ok, document} = Req.get!(url).body
                    |> Floki.parse_document()
  document
end

def grab_json(html, fnames, url) do
  html
  |> Floki.attribute("script", "src")
  |> Enum.find(fn str -> 
      Enum.any?(fnames, fn fname ->
      String.contains?(str, fname) 
    end)
  end)
  |> fetch_json(url)
end

defp fetch_json(nil, _), do: nil
defp fetch_json(path, url) do
  Req.get!(URI.encode(url <> path)).body
end

  

We grab the search.html, parse the HTML using Floki, look for script tags that contain the file_name and then download the that JavaScript File. Pretty straight forward so far!

T: JSON

While reading this next section is it important to note that given the constraints I think the Elixir Team did an absolutely incredible job with ExDoc. Engineering is about working within constraints and they killed it. You would be hard-pressed to find any language that gives you machine-readable docs at all for any reason.

Here is where things get a little in the weeds, remember when I said different versions of ExDoc can produce different versions of that JavaScript? Well, it also hasn’t always produced “standards” compliant JSON. Thanks to the incredible engineering of Browsers and JavaScript engines, that doesn’t matter. In our case Jason expects to work with standards compliant JSON… Suffice to say ExDoc has been patched and a bunch of the following wouldn’t be necessary with latest ExDoc generated docs.

First we gotta cleanup all the JavaScript from around the JSON that we’re after:

    def cleanup(<<"<", _::binary>>), do: "" # Html or something
def cleanup(nil), do: ""
def cleanup(str) do
  str = str
  |> String.trim()
  |> String.trim_leading("searchNodes=") 
  |> String.trim_leading("searchData = ")
  |> String.trim_leading("searchData=")
  |> String.trim_leading("searchNodes = ") 
  |> String.trim_leading("sidebarNodes=") 
  |> String.trim_leading("sidebarNodes = ") 
  |> String.trim_leading("var versionNodes = ")
  |> String.trim_leading("var versionNodes=")
  |> String.trim_leading("versionNodes=")
  |> String.trim_leading("versionNodes = ")
  |> String.trim_trailing(";\nfillSidebarWithNodes(sidebarNodes);")
  |> String.trim_trailing(";")
  |> String.trim()
  |> cleanup2()
end

  

This is pretty self-explanatory, over a decade the JS has had many forms and come with many functions. This next part is a little strange…

    def cleanup2(str) do 
  case Regex.named_captures(~r/\<\<(?<binary>[\d\,\s]*)...\>\>/, str) do
    nil -> str
    %{"binary" => bin} -> 
      ha = bin
      |> String.trim()
      |> String.trim_trailing(",")
      |> String.split(", ")
      |> Enum.map(fn str -> <<String.to_integer(str)>> end)
      |> Enum.join("")

      Regex.replace(~r/\<\<([\d\,\s]*)...\>\>/, str, "\""<>ha<>"\"")
  end
end

  

Remember when I mentioned that the JSON isn’t spec compliant? Well this is an example where certain types of binary strings weren’t being encoded correctly, I am sharing it here for posterity, but its been fixed.

I am not proud of what happens next but ExDoc has a hand built JSON Encoder, and for most of its existence it encoded the values by using the Inspect protocol. If everything lines up correctly, Jason will work, if not you need to parse it manually.

    def try_decode(nil), do: nil
def try_decode(""), do: ""
def try_decode(str) do
  str = cleanup(str)
  try do
    Jason.decode!(str)
  rescue
    e ->
      Logger.error(e)
      try do
        decode!(str)
      rescue
        e ->
          Logger.error(e)
          str
      end
  end
end

  

So this is the part of the post where I hand rolled a JSON Decoder. And because I feel the need to explain the nested try rescues’, sometimes when you write some ugly code you want it to look ugly to match that.

First we try Jason.decode!, which works most of the time and is excellent, and should work for all new packages! If that fails, then we try my handle rolled decode! function. And if that fails, it’s not going to be indexed.

I won’t get into the meat of the functions, it’s here, but it tokenizes the JSON using a regex, and then recursively walks it into Lists and Maps. Which might feel anti-climatic but I don’t recommend anyone do this unless forced to do this for a simple for fun project.

L: SQLite

    schema "packages" do
  field :meta, :map
  field :name, :string
  field :docs_html_url, :string
  field :downloads_all, :integer, default: 0
  field :downloads_day, :integer, default: 0
  field :downloads_recent, :integer, default: 0
  field :downloads_week, :integer, default: 0
  field :latest_docs_url, :string
  field :html_url, :string
  field :latest_stable_version, :string
  field :last_pulled, :utc_datetime

  field :search_items_json, {:array, :map}
  field :sidebar_items_json, :map
  field :docs_config_json, {:array, :map}

  timestamps()
end

  

Here is the Ecto Package schema. We won’t dig too much into this, but we also grabbed some metadata that might be relevant later, and 2 other JSON objects that might come in handy later.

An important aspect of any ETL pipeline is that during the Load stages, you want to store as much data as you can within the constraints of your system. In this case, once we’ve downloaded all the data AND created an FTS index the SQLite db filesize is under 500mb, which is totally fine. And you never know what data might come in handy in the future:

a new design comes down that requires us to show downloads
turns out sidebar_items has cleaner data for search
we want to update our index and only want packages that haven’t been updated recently

Some might argue to only store the minimum, that’s true for sensitive or extremely tight memory budgets, but in this case it’s neither! Alternatively we could have only created columns for data we care about, and have columns that store the raw JSON data.

Indexing

This is the part of the post where all the “hard” stuff is behind us, in retrospect I wouldn’t have estimated that simply collecting the data would have been 90% of the time spent and that is because ETL work is very difficult to estimate. In the “Real World” you really have to pad your estimates when working with outside data. Sometimes you get data in clean, standards compliant format with libraries to parse it, other times you get a CSV exported from Excel 97 with unknown quoting.

We have the data, it is loaded up into SQLite and cleaned up, lets make an index! I’ve gone into this a little bit before but here is my migration:

    def up do
  execute """
    CREATE VIRTUAL TABLE packages_index USING fts5(
      doc,
      title,
      type,
      ref UNINDEXED,
      package_id UNINDEXED,
    tokenize="porter")
  """
end

def down do
  drop table("packages_index")
end

  

I wish I had a better answer here for why I went with the porter tokenizer but I don’t, after a little experimentation I went with this one and it worked. You will need to make sure to keep this schema PackageIndex in sync with Packages but that’s not so bad if you are using context’s.

Searching

Just using this the default way ends up with pretty poor search results. A good example is trying out “Ecto.Query.preload” using the basic syntax:

    from(i in PackageIndex, where: fragment("packages_index MATCH ?", ^"Ecto.Query.preload", order_by: [asc: "rank"]))

  

You will find that packages that use Ecto.Query.preload will come up before Ecto.Query.Preload because they reference this exact string more often than the Ecto.Query doc’s do. This won’t do for what I want:

Better Search

    def search(term) do
  term =
    term
    |> String.trim()
    |> String.trim_trailing(".")
    |> String.replace(" " , "+")

  term_quoted = "\"#{term}\""

  base = from(i in PackageIndex,
    select: %{ i |
      function_rank: fragment("bm25(packages_index, 10.0, 5.0, 1.0)"),
      module_rank: fragment("bm25(packages_index, 5.0, 10.0, 1.0)"),
    },
    where: fragment("packages_index MATCH ?", ^term_quoted)
  )

  preload_query = from(p in Package, select: [:id, :name, :latest_stable_version, :docs_html_url])

  # See Part 2
end

  

SQLite FTS5 is a sharp tool if you query something and don’t manually quote it, it will blow up with incomprehensible errors. We also want spaces to be inclusive so we replace them with +. I stripe off ending . because it will return no results if there is an ending period.

Then we set up a base query, with a function_rank, module_rank and a where clause limiting us to the search term. SQLite uses the bm25 algorithm for estimating a documents relevance based on a query. In this case, the function_rank weighs the the function name column higher than all the rest. The same for the module column in module_rank. And then we setup a preload function to only query the fields we want, and leave out all the JSOn.

    # part 2
q = from(i in subquery(base), 
  join: p in assoc(i, :package),
  select: %{ i |
    rank: fragment("
                    CASE
                      WHEN (lower(?) = lower(?) and lower(?) = lower(?)) THEN -5000.0
                      WHEN ? = ? and (lower(?) = lower(?) or lower(?) = lower(?)) THEN -2000.0
                      WHEN ? = ? and ? = ? and instr(lower(?), lower(?)) = 1 and abs(length(?) - length(?)) < 3 THEN -1000.0
                      WHEN ? <> 'module' and instr(lower(?), lower(?)) = 1 and abs(length(?) - length(?)) < 3 THEN -500.0
                      WHEN instr(lower(?), lower(?)) = 1 THEN -30.0 
                      WHEN instr(lower(?), lower(?)) = 1 THEN -20.0 
                    ELSE 0.0
                    END + ?  + ? + ?",
      p.name, ^term, 
      i.title, ^term, 

      i.type, "module",
      p.name, ^term, 
      i.title, ^term, 

      i.type, "module",
      p.name, ^term, 
      i.title, ^term, 
      i.title, ^term, 

      i.type,
      i.title, ^term,
      i.title, ^term,

      p.name, ^term,
      i.title, ^term,
      "function_rank", "module_rank", i.rank
    )
  }
)

q2 = from(i in subquery(q), 
  select: %PackageIndex{
    id: i.id,
    title: i.title,
    type: i.type,
    package_id: i.package_id,
    ref: i.ref,
    rank: i.rank
  },
  order_by: [asc: i.rank],
  limit: ^limit
)
q2
|> Repo.all()
|> Repo.preload(package: preload_query)

  

Here is where the meat of the “better search” query is, in a massive CASE statement. I won’t go into every term, but basically we will boost packages that meet certain criteria from most to least:

Exact Matches on name or title
Module exact matches
Query is in the package name
…etc

Then I sum that extra “boost” with the built in rank that FTS5 gives when using the index and the function_rank and module_rank we calculated above. This gives us the base query that I can sort and limit on, while also boosting the VERY close matches. Which is exactly what we do in the final query. The end results is reasonable, and near instant queries for as you type.

Go check it out https://hex-search.fly.dev/!

This code is kinda ugly because it is the output of trial and error. I am deliberately not refactoring some of this so you can see that even experienced developers get frustrated and throw stuff at the wall hoping it would work.

The first version of this query would take entire seconds to search and still came up with terrible results. This kind of goes for all search, no matter what tool you end up using you will end up tinkering with it forever. Users will find a query that “should just work” because expectations are set by Google.

Discussion

My goal with this post was to walk you through my experiences while scratching a personal itch. Real World projects always end up messier than you’d like and that’s just life.

Please reach out and help make this better! Pull Requests are welcome and let me know if it ends up being useful to you, or roast me about how I missed a completely obvious way to approach this problem!

Sorting and Deleting many-to-many assocs with Ecto and LiveView

2023-07-25T00:00:00+00:00

In this post, we’ll use Ecto’s :sort_param and :delete_param options, along with LiveView, to sort and delete elements within a many-to-many association. Fly.io is a great place to run your Phoenix LiveView applications! Check out how to get started!

Ecto enables us to effortlessly work with different types of associations. In a many-to-many relationship, we can easily insert, modify, or delete elements. However, what if we want to sort the elements in a specific order? How can we remove specific records from an association?

Thankfully, Ecto has two new options to make that easier! When dealing with associations or embeds using cast_assoc/3 and cast_embed/3, respectively, we can use the :sort_param and :drop_param options. These options enable us to save associations in a specific order or delete associations based on their position.

But wait, there’s more! Passing these new parameters from LiveView is incredibly straightforward. In this post, we will leverage the power of LiveView and Ecto to sort and delete elements in a many-to-many relationship. Here’s what we’ll do:

Define the necessary Ecto.Schemas.
Define the changesets that enable us to use the :sort_param and :drop_param options.
Set up a form with the required inputs to populate a many-to-many association.
Incorporate the magic of checkboxes into our form to sort and delete elements.

By the end, we will have achieved something like this:

Defining the Ecto Schemas

Let’s begin by defining our example. We’ll take inspiration from a bookstore scenario, where a book can have one or many authors, and an author can write one or many books. To represent this, we’ll define three schemas: books, authors, and author_books.

First, we define the author_books schema, which serves as the intermediate table in our many-to-many relationship:

    defmodule ComponentsExamples.Library.AuthorBook do
  use Ecto.Schema
  import Ecto.Changeset

  alias ComponentsExamples.Library.{Author, Book}

  schema "author_books" do
    field :position, :integer
    belongs_to :author, Author, primary_key: true
    belongs_to :book, Book, primary_key: true
    timestamps()
  end
end

  

This schema is straightforward. It has two relations, :author and :book, to preload information about the author and book, respectively. Additionally, note the :position field. This field helps us store the author’s position, enabling us to preload the authors of a book in the order specified by their positions. We’ll explore this in more detail shortly.

Now we define the books schema:

    defmodule ComponentsExamples.Library.Book do
  use Ecto.Schema
  import Ecto.Changeset

  alias ComponentsExamples.Library.AuthorBook

  schema "books" do
    field :price, :decimal
    field :publication_date, :utc_datetime
    field :title, :string

    has_many :book_authors, AuthorBook, 
      preload_order: [asc: :position], 
      on_replace: :delete

    has_many :authors, through: [:book_authors, :author]
    timestamps()
  end
end

  

The books schema consists of three fields: title, price, and publication_date. We have also defined two associations that work neatly together.

The first association, has_many: book_authors, enables us to save and query the relationship between books and authors using the AuthorBook schema. Two essential options have been included:

The :preload_order option ensures authors are sorted based on the earlier mentioned :position field, in ascending order, when preloaded.
The on_replace: :delete setting ensures that any association element not included in the parameters sent will be deleted.

It’s important to note that the option on_replace: :delete is required for the sort and delete operations, and here’s why it makes sense: Since the client is modifying and deleting children, it becomes necessary for them to provide the full listing. Any elements missing from the listing are considered discarded because there is no way to know if the “old” ones should be preserved or not.

The second association, has_many: authors, enables us to conveniently preload the authors of a book. We achieve this by using the associations we mentioned earlier: [:book_authors, :author]. The :book_authors association guarantees that the authors are already listed according to their positions. It’s a good trick, isn’t it?

Next, we need to define the author schema, which, for this example, does not have any specific constraints:

    defmodule ComponentsExamples.Library.Author do
  use Ecto.Schema
  import Ecto.Changeset

  alias ComponentsExamples.Library.AuthorBook

  schema "authors" do
    field :bio, :string
    field :birth_date, :string
    field :gender, :string
    field :name, :string
    timestamps()

    many_to_many :books, AuthorBook, join_through: "author_books"
  end
end

  

With this, we have defined the necessary schemas to model the bookstore scenario.

Using :drop_param and :sort_param

Let’s start by defining the Book.changeset/3 function, to create or modify a book and its associated authors:

      def changeset(book, attrs) do
    book
    |> cast(attrs, [:title, :publication_date, :price])
    |> validate_required([:title, :publication_date, :price])
    |> cast_assoc(:book_authors,
      with: &AuthorBook.changeset/3,
      sort_param: :authors_order,
      drop_param: :authors_delete
    )
  end

  

Within this function, we specify the names of the parameters used for sorting and deleting elements within the :book_authors association: :authors_order and :authors_delete. Additionally, we use the :with option, which accepts a function with arity 3 to create the child records. The third argument passed to the function contains the position of each child element.

Warning: Make sure that you are using Ecto v3.10.2 or a newer version. If you attempt to send a function with arity 3 in the :with option on older versions, you may encounter an error.

Now, let’s save the position in our :author_books table. To accomplish this, we define the AuthorBook.changeset/3 function:

      def changeset(author_book, attrs, position) do
    author_book
    |> cast(attrs, [:author_id, :book_id])
    |> change(position: position)
    |> unique_constraint([:author, :book], name: "author_books_author_id_book_id_index")
  end

  

In this function, we ensure to handle the position passed as the third argument to our changeset function. We cast the attributes :author_id and :book_id, then modify the changeset to include the position. Finally, we enforce a unique constraint on the combination of :author_id and :book_id using the unique_constraint function.

With this, our work with Ecto.Changeset and Ecto.Schema is complete. Now let’s explore how to send :authors_order and :authors_delete from a form.

Using `inputs_for` to populate a has_many association

In this example, we assume that there are existing authors in our database, and we need to provide the user with options to choose the author(s) of a book. To achieve this, we first create a book changeset with at least one book_author in the :book_authors association. We then build a form using this changeset.

Let’s prepare our assigns:

      def update(%{book: book} = assigns, socket) do
    book_changeset = Library.change_book(book)

    socket =
      socket
      |> assign(assigns)
      |> assign_form(book_changeset)
      |> assign_authors()

    {:ok, socket}
  end

  

The crucial part is the assign_form/2 function, where we build a form for the Book. If the book does not have any author in the :book_authors association, we create an empty %AuthorBook{} and include it as a single author in the book’s :book_authors association —this allows us to render at least one input to fill in the author information. Finally, we convert this modified changeset into a form:

      defp assign_form(socket, %Ecto.Changeset{} = changeset) do
    if Ecto.Changeset.get_field(changeset, :book_authors) == [] do
      book_author = %AuthorBook{}
      changeset = Ecto.Changeset.put_change(changeset, :book_authors, [book_author])
      assign(socket, :form, to_form(changeset))
    else
      assign(socket, :form, to_form(changeset))
    end
  end

  

To provide the user with a selection of authors from the database, we’ll use a select input. The select input expects a list of options in the format {label, value}. We assign these options using the assign_authors/1 function:

    defp assign_authors(socket) do
  authors =
    Library.list_authors()
    |> Enum.map(&{&1.name, &1.id})

  assign(socket, :authors, authors)
end

  

In this function, we fetch the authors from the database using Library.list_authors(). We then transform each author into a {label, value} tuple, where the label represents the author’s name and the value corresponds to the author’s ID. Finally, we assign the authors list to the socket for use in the template.

With our assigns prepared, we can now define the inputs to send the attributes of the :book_authors association.

    def render(assigns) do
  ~H"""
  <div>
    <.simple_form 
      for={@form} 
      phx-change="validate" 
      phx-submit="save" 
      phx-target={@myself}
    >
      ...
      <div id="authors" phx-hook="SortableInputsFor" class="space-y-2">
        <.inputs_for :let={b_author} field={@form[:book_authors]}>
          <div class="flex space-x-2 drag-item">
            <.icon name="hero-bars-3" data-handle />
            <.input
              type="select"
              field={b_author[:author_id]}
              placeholder="Author"
              options={@authors}
            />
          </div>
        </.inputs_for>
      </div>
      <:actions>
        <.button phx-disable-with="Saving...">
          Save
        </.button>
      </:actions>
    </.simple_form>
  </div>
  """
end

  

We use the function component <.inputs_for> to populate the @form[:book_authors] association. Within this component, we access the attributes of each individual :book_authors using the b_author variable.

We create a select input to populate the b_author[:author_id] field, passing in the list of authors as available options.

With this, we are now able to send information about books and authors. However, we still need to make one final modification to send our :sort_param and :delete_param parameters.

Checkboxes magic

:drop_param and :delete_param require a list of numerical indexes to identify the elements that need to be reordered or deleted. To send this list of indexes, we need to include the position of each b_author element.

To do this, we add a hidden input inside the <.inputs_for> component for each b_author. This hidden input will send the value of the element’s position using value={b_author.index}:

  def render(assigns) do
    ~H"""
    <div>
      <.simple_form 
        for={@form} 
        phx-change="validate" 
        phx-submit="save" 
        phx-target={@myself}
      >
        ...
        <div id="authors" phx-hook="SortableInputsFor" class="space-y-2">
          <.inputs_for :let={b_author} field={@form[:book_authors]}>
            <div class="flex space-x-2 drag-item">
              <.icon name="hero-bars-3" data-handle />
+             <input 
+               type="hidden" 
+               name="book[authors_order][]" 
+               value={b_author.index} 
+             />
              <.input
                type="select"
                field={b_author[:author_id]}
                placeholder="Author"
                options={@authors}
              />
            </div>
          </.inputs_for>
        </div>
        <:actions>
          <.button phx-disable-with="Saving...">
            Save
          </.button>
        </:actions>
      </.simple_form>
    </div>
    """
  end

The naming convention of this input is crucial. As the form is built from a Book changeset, the input names reflect this structure name. For our example, each input name starts with “book” followed by square brackets and the attribute name being filled, such as name="book[publication_date]".

It’s important to note that this convention holds true unless you have specifically set up the :as option of the to_form/2 function and used a different prefix for the form inputs. For example: to_form(changeset, as: "my_form")

In the case of :authors_order, which involves multiple inputs, each element is uniquely identified by appending an index. For instance, the name attribute would be represented as name="book[authors_order][]".

With this additional input, we can now manage the position of each element. Let’s see how we can add and remove elements from the assoc:

  def render(assigns) do
    ~H"""
    <div>
      <.simple_form 
        for={@form} 
        phx-change="validate" 
        phx-submit="save" 
        phx-target={@myself}
      >
        ...
        <div id="authors" phx-hook="SortableInputsFor" class="space-y-2">
          <.inputs_for :let={b_author} field={@form[:book_authors]}>
            <div class="flex space-x-2 drag-item">
              <.icon name="hero-bars-3" data-handle />
              <input type="hidden" name="book[authors_order][]" value={b_author.index} />
              <input 
                type="hidden" 
                name="book[authors_order][]" 
                value={b_author.index} 
              />
+             <label>
+               <input
+                 type="checkbox"
+                 name="book[authors_delete][]"
+                 value={b_author.index}
+                 class="hidden"
+               />
+               <.icon name="hero-x-mark" />
+             </label>
            </div>
          </.inputs_for>
        </div>
        <:actions>
          <.button phx-disable-with="Saving...">
            Save
          </.button>

+          <label class="block cursor-pointer">
+            <input 
+              type="checkbox" 
+              name="book[authors_order][]" 
+              class="hidden" 
+            />
+            <.icon name="hero-plus-circle" /> add more
+          </label>
        </:actions>
      </.simple_form>
    </div>
    """
  end

To add elements to the book[authors_order][] association, we use another checkbox input hidden inside a label with a plus icon. This input is essential to append new elements to the end of the list. It is placed outside the <.inputs_for> function and after it. These checkboxes send the index of each b_author element as the value.

For deleting elements, we need to send the index of the element to be removed. Inside the <.inputs_for> function, we add another checkbox input hidden inside a label with a hero-x-mark icon. These checkboxes also send the index of each b_author element as the value.

Let’s see how the information of these checkboxes is sent when the validate or submit events are triggered:

    [debug] HANDLE EVENT "validate" in ComponentsExamplesWeb.LibraryLive
  Component: ComponentsExamplesWeb.MulfiFormComponent
  Parameters: %{"_target" => ["book", "authors_order"], "book" => %{"authors_order" => ["0", "on"], "book_authors" => %{"0" => %{"_persistent_id" => "0", "author_id" => "2", "book_id" => "2", "id" => "88"}}, "price" => "2323", "publication_date" => "2023-07-13T19:37", "title" => "Libro prueba"}}
[debug] Replied in 982µs

  

If we focus on the authors_order attribute, we can see that a list ["0", "on"] is sent. This list indicates that there are two authors, with indexes "0" and "on"… wait, what? Well, this weird “on” index is used to identify when we’ve just added a new item using our “add more” checkbox.

What if we have another checkbox, similar to the one we used to add new authors, but placed before the <.inputs_for> component? In that case, when adding an element, the "on" parameter would be at the beginning of the list: ["on", "0"].

Let’s now see what happens when we delete an element:

    [debug] HANDLE EVENT "validate" in ComponentsExamplesWeb.LibraryLive
  Component: ComponentsExamplesWeb.MulfiFormComponent
  Parameters: %{"_target" => ["book", "authors_delete"], "book" => %{"authors_delete" => ["1"], "authors_order" => ["0", "1"], "book_authors" => %{"0" => %{"_persistent_id" => "1", "author_id" => "1"}, "1" => %{"_persistent_id" => "0", "author_id" => "2", "book_id" => "2", "id" => "88"}}, "price" => "2323", "publication_date" => "2023-07-13T19:37", "title" => "Libro prueba"}}
[debug] Replied in 944µs

  

In this example, the authors_delete attribute is sent along with a list of indexes representing the elements to be deleted.

You may have noticed that both example logs include a _persistent_id. This identifier serves as internal Phoenix book keeping to track our inputs effectively.

Awesome! We’ve got it all covered now! Adding and deleting elements from the association is a breeze. Just send the required info from our form, and our changesets and schemas will take care of the rest. Way to go!

(For a complete view of the code, please refer to this PR.)

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Some Things to Keep in Mind

When using this approach to sort and delete items in a association, there are a few considerations worth noting:

Make sure to preload the association elements before performing any updates. This ensures that the data is correctly ordered.
It’s important to be aware that this approach relies on preloaded elements. If you want to order associations in a relationship but haven’t preloaded all the elements, manual handling of reordering will be necessary.
While this example demonstrates the manipulation of a many-to-many relationship, you can easily adapt these concepts to work with simpler relationships like has-many.

Elixir Docs are Built Different

2023-07-06T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

A few common refrains from developers who are new to Elixir are:

“Where are the blog posts?”
“Stack Overflow?”
“Google was no help…”

And I totally get it, if you come from another language ecosystem, you would be right to prioritize tutorials or blog posts first. It is just way more likely you get a useful snippet than if you read the official docs, if you can find them at all.

In Elixir that is simply not the case, our documentation is not only fairly easy to navigate, it also includes copious amounts of discussion, guides and examples in the docs for a Module.

Map

Lets dive right in to an example. Say we are going to use a Map data structure in Elixir. If we go to the official documentation for Map we will find almost two whole pages of discussion on what a map is, how to use it, warnings on its usage and with extensive examples. Further, if we search on Google “elixir map” this page will be the top result.

If we continue scrolling, you will find any type information that’s defined. Then a list of every function defined in that module, with examples and type specs for each! Crucially, each example also shows what the expected output would be. These are also called Doctests and executed as tests. Take a step back and appreciate that for a second, you know these examples work because it would break the build if they didn’t! Further, you can write Doctests in your own code and execute them as tests as well!

If we click the little </> icon next to its name (at the top right of the above screenshot) we get brought to the definition in GitHub!

Because Elixir is an immutable and functional language, what you see is what you get. Nine times out of ten, if the documentation sucks, we can simply read the code to get a better idea of what’s going on. I should mention, if we’re using a library that has a ton of macros, this will get harder to navigate. And this isn’t just Elixir the language that is like this, every library on hex.pm has auto-generated documentation hosted on hexdocs.

Hex

Hex is another valuable resource for Elixir developers, below is an annotated screenshot from one of my favorite libraries, jason.

A. Takes you to the online hosted docs. Usually hexdocs, but developers can host it or configure it however they want. B. Links to that version’s documentation. C. Links to the code diff between that version and the previous one. More on this in a second. D. Lists every package that depends on this library, in order of downloads. This can be a good indicator that it is a solid package.

In this one page, we have access to everything about this library and the ability even review the code before we upgrade. Here is the diff between version 1.3.0 and 1.4.0 for jason.

This is incredible, with a single click we can review a package, and its changes entirely in one page. If you direct your attention to the URL you will see 1.3.0..1.40 we can change those to see the diff between any version. So if we skipped a few versions, we can review it before accepting the change blindly!

Wrapping up

A big part of learning any new language is learning how to “read the manual”. In the case of Elixir, developer documentation has always been a top priority, with an ongoing effort to improve it. I am going to list off a couple of the “Greatest Hits” when it comes to documentation, and I hope you enjoy!

Ecto: these docs are the most used page for me in all of hexdocs, partially because it is a huge library that does a ton, but also because It’s just so full of useful knowledge.
Phoenix Guides: these tutorials are actually written in the Phoenix repository right alongside the code.
NimbleParsec: this is a library for generating fast text-based parser combinators, if you don’t know what that is simply read the doc! I had no idea what they were before I read these docs, and they turn out to be incredibly helpful!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

for Can Do More

2023-06-26T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Elixir has a for special form called a “list comprehension” and not enough people know what it can do. It can do so much more than a simple for loop, it is kind of like if the Enum and Stream modules got together and had a macro. And I want you to know about it!

Here is a standard Elixir example, reading from a file, mapping over it, filtering over it, converting it into a map:

    File.read!("my_lines.txt") # key=val\n
|> Enum.filter(fn str -> "" == str end)
|> Enum.map(fn line -> 
      [key, val] = String.split(line, "=", trim: true)
        {key, val}
    end)
|> Enum.into(%{})

  

This is standard Elixir, clearly breaking each step into a pipeline flow, doing what you expect. One downside is that each step of this iteration creates a copy of the list with the changes applied. In most cases this is totally cool, the Erlang VM will garbage collect this with ease.

But, if my_lines.txt ends up being a huge file we might have a problem with memory. Next we break that into Stream:

    File.stream!("my_lines.txt") # key=val\n
|> Stream.filter(fn str -> "" == str end)
|> Stream.map(fn line -> 
      [key, val] = String.split(line, "=", trim: true)
        {key, val}
    end)
|> Enum.into(%{})

  

In programming parlance a Stream is what we call a “Lazy” container, meaning until we call Enum.into(%{}) in our above code, the code is not executed. The Enum functions are eager and will immediately execute. A Stream is simply a struct that builds up a list of operations to apply to a list, when executed it iterates through the input list, Range, Enumerableor functions that create a stream.

If we were to reimplement using for:

    for line <- File.stream!("my_lines.txt"), line != "", into: %{} do
  [key, val] = String.split(line, "=", trim: true)
    {key, val}
end

  

The first line accomplishes most of the work in this code so let’s break it down:

line <- File.stream(..), is the source of iteration you can tell because it has that funky left pointing arrow, in our case streaming the file. for is eager, so it will fully execute the stream similar to Enum does.
The next argument after our iteration, line != "" is a guard function this is equivalent to our Enum.filter above.
The final argument, into: %{}, lets us use the Collectable protocol to roll the result into a map. It’s functionally equivalent to Enum.into above.

The beauty of this statement is that it works like the Stream in that it does one iteration over one list , and we don’t need to mix and match Enum/Stream functions. We can even use a Stream to start it! One major downside is in clarity, this line is verbose and can be less clear than our pipeline of operations.

Let’s reduce

We could have also used the reduce keyword here instead of into like so:

    for line <- File.stream!("my_lines.txt"), line != "", reduce: %{} do
  acc ->
      [key, val] = String.split(line, "=", trim: true)
        Map.put(acc, key, val)
end

  

The key change is that after the do block, we need to have a var_name -> right-hand arrow that names the accumulator and returns a new accumulator. This example isn’t an amazing use of reduce, but it lets us have fine-grained control over how our accumulator is updated.

More?

We can also do multiple iterators, like so:

    for x <- [1, 2], y <- [2, 3] do
    x * y 
end 
# [2, 3, 4, 6]

  

This one is straight from the Elixir Docs, which is a testament to how frequently I use this feature. I suspect it might be handy when doing code challenges or to implement an advanced FizzBuzz! Let me know if you have a great example!

Discussion

The for special form is an invaluable tool for iteration and should not be overlooked!. That said, it’s also a bit of a sharp blade… Able to produce truly unreadable lines of code for future coders, while making you feel like a genius.

Bonus: Recursion

Elixir is also superb at recursion, in fact the Erlang Community uses it much more often than we do, though it does require rethinking how we might write it! So here it is:

    def parse(), do: parse(File.read!("my_lines.txt"), %{})
def parse([], acc), do: acc # End Condition
def parse(["" | rest], acc), do: parse(rest, acc) # Skip empty
def parse([line | rest], acc) do # Default Case
  [key, val] = String.split(line, "=", trim: true)
    parse(rest, Map.put(acc, key, val))
end

  

We lean pretty hard on pattern matching here, and the best way to read this from top to bottom:

Read the text into a list of lines and set up an empty map as our accumulator
if the list is empty, we’re done, and we return the accumulator
if the line is empty, ignore that line and continue parsing, could have also used a guard here.
finally parse the item, add it to the accumulator and continue parsing.

This is called “tail recursion” because it does the recursion at the “tail” or “end” of our function. The VM optimizes this into a what is effectively a loop to avoid infinite stacks. You might be wondering why the Erlang Community maybe prefers this style?

For one thing it’s cultural, people learn to do it, they do it that way, they teach it that way. This is always the case for any norm.
It gives fine-grained control over what allocation happens, how you filter/map/reduce, you don’t need any of those concepts you just write code that does the thing.
Finally, it is that they don’t have do/end or def so their function heads are less verbose.

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Using Ctrl+Enter to submit a Text Area with LiveView

2023-06-20T00:00:00+00:00

In this post, we’ll explore how to use hooks and a few lines of JavaScript to trigger form submission simply by hitting Ctrl+Enter keys within a text area. Fly.io is a great place to run your Phoenix LiveView applications! Check out how to get started!

Problem

You have a text area for sending long messages. Users should be able to submit without moving their hands from the keyboard. Pressing Enter only creates a new empty line instead of sending the message:

Solution

You can define a Hook that listens for the keydown event and checks if a specific key —or key combination— is pressed. This hook can then be used to trigger an event on a DOM element, like a form.

For this example, in your app.js file, define a hook that automatically triggers the submit event for the form when you press Ctrl+Enter:

    Hooks.CtrlEnterSubmit = {
  mounted() {
    this.el.addEventListener("keydown", (e) => {
      if (e.ctrlKey && e.key === 'Enter') {
        this.el.form.dispatchEvent(
          new Event('submit', {bubbles: true, cancelable: true}));
      }
    })
  }
}

  

By setting the bubbles option to true, you are allowing the event to propagate through the DOM tree until it reaches the window listener. This enables LiveView to recognize the submit event. Additionally, by using the cancelable option, you prevent Firefox from submitting the form via HTTP.

You have the option to choose a different key combination. For example, you can submit the form by simply pressing Enter, while still having the option to create a new line by using Shift+Enter:

    Hooks.CtrlEnterSubmit = {
  mounted() {
    this.el.addEventListener("keydown", (e) => {
      if (e.key == "Enter" && e.shiftKey == false) {
        this.el.form.dispatchEvent(
          new Event('submit', {bubbles: true, cancelable: true}));
      }
    })
  }
}

  

Now, render the text area input, and add the phx-hook option:

    <.input 
  field={@form[:message]} 
  type="textarea" 
  placeholder= "New message" 
  phx-hook="CtrlEnterSubmit" 
/>

  

It works!

Discussion

With hooks, the possibilities are endless. You can listen for different key combinations to trigger form events —such as submitting a form with phx-submit or capturing changes with phx-change—, but that’s not all! You can also use different listeners like focusout, input, or keyup to trigger those events. It’s an exciting world of possibilities, isn’t it?

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

A LiveView is a Process

2023-06-14T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. This post is about Elixir Processes and how they work as Phoenix LiveViews. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

LiveView lets users get up to speed quickly by being easy to learn, especially with the familiar mount, render and event handler system that React developers will immediately recognize. But LiveView differs from other front-end frameworks in one very consequential way:

A LiveView is a process.

And even though the LiveView docs immediately call out that fact:

A LiveView is a process that receives events, updates its state, and render updates to a page as diffs.

This fact sets LiveView apart from almost every other front-end framework and is so important that I think it could use a little reiteration.

Elixir has Processes

In Elixir and Erlang, ‘processes’ do not refer to operating system processes or threads. They are what’s sometimes called green threads or actors. On a single core CPU they run concurrently and are scheduled and managed by the Erlang Virtual Machine; on multiple cores they run in parallel. Each process in Elixir and Erlang needs ~300 words of memory, and takes microseconds to start, so they are incredibly cheap. In the Erlang Virtual Machine, everything that executes code is running in a process.

I highly recommend you check out this excellent guide from the Elixir website that goes into some of the details about processes. We’re going to keep it slightly higher level here.

Each Process can execute code, it has a first-in-first-out mailbox that any other process can send messages to, and it can also send messages. Each process is sequential, meaning it can only handle one message at a time. The Erlang VM operates kind of like an Operating System scheduler, where it can start and pause or “preempt” work whenever it wants. While it is waiting for a message, your process is completely ignored by the scheduler and doesn’t burn up precious resources.

When we work with GenServers, which are higher level abstractions on top of processes, our flow looks kind of like this:

A process starts and sets its initial state, then it waits for a message. When it receives a message it handles it, gets a new state and returns to waiting. This is an important thing to understand: a process is only able to execute if it receives a message. When it is started or initialized a process executes some code, then it waits for a message. This means an idle process won’t consume resources. There are also some messages that are handled internally by GenServers for you, but that’s outside the scope for this post.

Since an individual process is sequential, if the message handling function takes a long time to execute, the mailbox or queue may back up. If the process is not expecting a ton of new messages this can be okay. While using a Task it is okay to do a costly calculation because those processes don’t expect more messages. While in the case of a LiveView process, a user will see a page that’s unable to respond to events or render updates, this is bad.

The LiveView Process

Just like any other process, LiveView follows a specific lifecycle. Here’s a simplified flow chart to illustrate this:

Where assigns is our state and event is a special case of a message we make special callbacks for. Every user event or params event is a message being handled by our callbacks. So let’s think through some of the implications of this.

Every user has their own Process.

Despite each user having their own process, it’s not an issue due to the lightweight nature of these processes. The benefits we gain in terms of performance and scalability make it well worth it. To be clear: if you do a normal HTTP request using Phoenix controllers that connection gets its own Process too, it just is immediately killed once you’ve sent the response and closed it. In LiveView we keep that process alive.

LiveView lifecycle functions need to respond quickly.

Every message is handled sequentially, meaning we need to make sure that handle_event, mount, handle_params, and handle_info functions return quickly since they could be blocking a user interaction.

If you have some slow job, query or calculation you should use the built-in async primitives, such as Tasks. Berenice wrote an excellent post showing an example of doing that just in her recent Async Processing in LiveView post.

Be careful of what you put into assigns.

Assigns are kept for the entire lifetime of the page for every user, and you can quickly chew up memory by shoving a ton of stuff into it. It is okay to re-query stuff you need, and look into using streams when you have a long list of items.

Wrap up

In conclusion, the process is the heart of the Erlang VM, making programming in Elixir a uniquely fast, resilient, and special experience. On purpose, this is only scratching the surface of what a Process is because if the Phoenix team is doing their job right, you shouldn’t need to know much more to be productive and effective at building scalable applications.

When you are ready to learn more, here are two great resources:

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Adding Dialyzer without the Pain

2023-06-12T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. This post is about how to add Dialyzer to an existing Elixir project without making your whole team hate you. In fact, they may even thank you. Fly.io happens to be a great place to run Elixir applications. Check out how to get started!

This post is about how to start using Dialyzer on an established Elixir project without losing your mind in the process. If you’ve been in the Elixir community for any length of time, you’ve probably heard about Dialyzer. It’s tool that does static code analysis to help spot bugs in our application before it goes to production. Dialyzer can also be finicky and the reported problems can be difficult to interpret. This all results in some mixed feelings about Dialyzer in the community.

Here, we assume you want the benefits of type checks but you don’t want to stop the project to clean up the 100s of legacy issues.

We’ll start with a little background, you can jump ahead and get started, if you prefer.

Why Check Types?

Erlang and Elixir are dynamically-typed, so why should we care about types? The nice part is that you aren’t strictly required to care about types at all. If you don’t want to write type definitions and check them with external tools, you don’t have to. However, many developers find some form of type specification useful as a form of documentation and a tool for preventing bugs. Luckily for us, Erlang and Elixir programmers are spoiled for choice when it comes to tooling that can help us through the use of types without needing to run our code first. The rest of this post will be an analysis of those tools as well as an example of how to implement a common Erlang/Elixir type-checking tool: Dialyzer.

The First Poll

A year ago, there was a poll posted to the Elixir Forum asking whether or not people used Dialyzer in their projects. Of the 207 votes (at the time of this post), about 72% of them used Dialyzer for at least some of their projects. The poll replies were where the real meat of the subject was; some highlights:

I find dialyzer to work well for projects, which are deployed. I tend to not like it for libraries, which don’t really use their code, because that usually means dialyzer doesn’t find half of the issues. - LostKobrakai

I have to admit that I do have quite a love/hate relationship with the Dialyzer. It does really catch potential errors, but the error messages can be really difficult to understand, especially when largish structs are involved. - xpg

No. Too slow, too cryptic. Hoping for something like Gradualizer to be usable. In the meantime, I invest time in end to end tests. - stefanchrobot

Dialyzer as it exists today is a net negative value add for me. I only use it if forced. In principle it checks out, but in practice it always costs more than it saves. - chrismccord (Creator of Phoenix)

Adding it into any project of reasonable size is slow, sometimes unsustainably so. - Qqwy (TypeCheck Core Team)

Admittedly, I’ve cherry-picked portions of responses that are critical of Dialyzer, but only to point out that Dialyzer is a controversial tool among Elixir developers.

The Second Poll

In order to get more opinions, I created a followup poll asking what tool (if any) people preferred to typecheck their Elixir/Erlang code. Poll options included:

In hindsight, I should have also added Gleam as an option.

As you can probably tell, we are fortunate to be spoiled for choice when it comes to type-checking in the Erlang/Elixir ecosystem, though of the 107 votes (at the time of this post), about 65% listed Dialyzer/Dialyzir as a typechecking tool they used with Elixir/Erlang. This seems pretty consistent with the previous poll. There weren’t as many replies as the original poll, but it still gave a good impression of the Elixir/Erlang type-checking landscape.

The Inspiration

Six months before making the followup poll, I watched a talk titled “Slaying the Type Hydra, or How We Went from 12,000 Dialyzer Errors to None”. In it, Jesper Eskilson describes how Klarna added Dialyzer into a preexisting system and how (at a high level) they integrated it into their CI and review process in order to steadily reduce their Dialyzer error count to zero. It was not an effortless/painless journey, but it was encouraging. I set out to try the same thing within my own company.

Adding Dialyzer to an Existing Project

As an exercise, let’s try to add Dialyzer (more specifically, Dialyxir) to some existing Elixir code. Remember that even though we may run mix dialyzer, we’re actually using Dialyxir as a convenience for Elixir projects. We’ll test out this process using a few large, well-known projects to see what we can learn: (ordered by GitHub stars at the time of this post)

Phoenix
~~Plausible Analytics~~
- Plausible already uses Dialyzer, good on them!
~~Supabase Realtime~~
- Supabase already uses Dialyzer, good on them!
~~Firezone~~
- Firezone already uses Dialyzer, good on them!
Livebook
Changelog
~~Membrane~~
- Membrane already uses Dialyzer, good on them!

Three seems like a good exercise, no? We’ll add Dialyzer To Phoenix, Livebook, and Changelog.

Add The Dialyxir Dependency

This is the easy part:

    # mix.exs
defp deps do
  [
    {:dialyxir, "~> 1.3", only: [:dev], runtime: false},
  ]
end

  

Followed by: mix do deps.get, deps.compile

Run Dialyzer

At this point, you’d normally run mix dialyzer, which would both generate your Dialyzer PLT files (more on those later) and then typecheck your project with them. We’re going to split up this step slightly for learning purposes.

Side Note: PLTs

We’ll talk more about this in Configuring CI, but Dialyzer operates using a file format called Persistent Lookup Table (PLT). Think of it as a cached Dialyzer run against a specific version of your Elixir/Erlang build.

To quote the Dialyxir docs:

Running the mix task dialyzer by default builds several PLT files:

A core Erlang file in $MIX_HOME/dialyxir_erlang-$OTP_VERSION.plt
A core Elixir file in $MIX_HOME/dialyxir_erlang-$OTP_VERSION-$ELIXIR_VERSION.plt
A project environment specific file in _build/$MIX_ENV/dialyze_erlang-$OTP_VERSION_elixir-$ELIXIR_VERSION_deps-$MIX_ENV.plt

NOTE: If you use the asdf language version manager, the location of $MIX_HOME will vary based on the current version of Elixir you are using: ~/.asdf/installs/elixir/$ELIXIR_VERSION/.mix

Generating PLTs

Running mix dialyzer --plt only generates the required PLT files to do typechecks, so let’s try that. Note that the times listed below are only for the time taken to generate the project PLTs, not the core Erlang/Elixir PLTs, as those are cached. This is not a performance benchmark by any means. Faster PLT generation does not equal better performance or higher-quality code. It’s dependent on the application, its dependencies, how many extra_applications it has configured, etc.

Time taken: (all times are done on the same machine, a Macbook Pro with an M2 Pro)

Phoenix: done in 0m18.76s
Livebook: done in 0m29.42s
Changelog: done in 0m38.99s

Now we can run mix dialyzer to perform the actual typecheck and see how many errors we find!

Phoenix:

Total errors: 173, Skipped: 0, Unnecessary Skips: 0
done in 0m0.84s
# ...a bunch of errors printed here...
done (warnings were emitted)
Halting VM with exit status 2

Livebook

Total errors: 27, Skipped: 0, Unnecessary Skips: 0
done in 0m3.37s
# ...a bunch of errors printed here...
done (warnings were emitted)
Halting VM with exit status 2

Changelog:

Total errors: 74, Skipped: 0, Unnecessary Skips: 0
done in 0m3.2s
# ...a bunch of errors printed here...
done (warnings were emitted)
Halting VM with exit status 2

Not bad, honestly! The first few work projects I ran this on had upwards of 400 errors. Regardless, these numbers can sometimes seem daunting. Who wants to fix 173 errors all at once? In my experience, no one. How can we make these errors easier to deal with? By ignoring them!

Ignoring Existing Errors

In the previous step, you may have noticed the “skips” and “unnecessary skips” given when running Dialyzer. This is actually a special feature of dialyxir that allows us to selectively ignore certain errors using a special flag to mix dialyxir: --format ignore_file. See the dialyxir docs on the subject for more details, but essentially all this does is generate one line for each error that already exists in a project. You can put these lines in a .dialyzer_ignore.exs file to make subsequent mix dialyzer runs ignore that particular error.

So, let’s try it out for each of our projects using mix dialyzer --format ignore_file | wc -l to count how many lines are generated.

Phoenix: 56
Livebook: 15
Changelog: 62

You’ll notice that the number of ignore lines doesn’t match the number of total errors from the previous step. This is because --format ignore_file will generate one line per error per entire file, not one line per specific error.

In the next release of dialyxir, a new --format ignore_file_strict will be added that improves on the original --format ignore_file pattern by allowing you to ignore specific errors in each file, even if the same error occurs multiple times in a file.

So, let’s copy each project’s ignore lines into a new file with a single list: .dialyzer_ignore.exs (you can configure the name of this file, but the default is fine for us)

    [
  # line 1...
  # line 2...
  # line 3...
  # ...
]

  

Rerunning mix dialyzer gives us:

Phoenix:

Total errors: 173, Skipped: 173, Unnecessary Skips: 0
done in 0m0.88s
done (passed successfully)

Livebook:

Total errors: 27, Skipped: 27, Unnecessary Skips: 0
done in 0m3.48s
done (passed successfully)

Changelog:

Total errors: 74, Skipped: 74, Unnecessary Skips: 0
done in 0m3.19s
done (passed successfully)

Hooray, the task passed successfully! Let’s commit our changes, push, and make a PR! We’ll talk more about configuring our CI to check for Dialyzer errors, but for now let’s assume that’s set up for us.

Preventing New Errors

With our new type-checking in place, let’s say someone makes a PR with an incorrect typespec on a new function in Phoenix:

    # lib/phoenix.ex

# Here our `@spec` says the function's return type is an integer:
@spec my_cool_func(integer()) :: integer()
def my_cool_func(an_integer) do
  new_integer = an_integer + 1

  # But the function is actually returning a string!
  Integer.to_string(new_integer)
end

  

Let’s see what happens when we run mix dialyzer again:

Total errors: 174, Skipped: 173, Unnecessary Skips: 0
done in 0m0.87s
lib/phoenix.ex:61:invalid_contract
The @spec for the function does not match the success typing of the function.

Function:
Phoenix.my_cool_func/1

Success typing:
@spec my_cool_func(integer()) :: binary()

_________________________________________________________________________
done (warnings were emitted)
Halting VM with exit status 2

We still ignore the other 173 errors, but now we’ve caught the new error!

Fixing Errors

Ignoring all our current errors is a great way to start using Dialyzer/Dialyxir, but eventually we’ll want to fix them. Let’s say we decide to ignore the above error by adding {"lib/phoenix.ex", :invalid_contract}, to the .dialyzer_ignore.exs file for the time being. Then we fix the error:

    @spec my_cool_func(integer()) :: String.t()
def my_cool_func(an_integer) do
  new_integer = an_integer + 1
  Integer.to_string(new_integer)
end

  

We’ll run mix dialyzer again and see:

Total errors: 173, Skipped: 173, Unnecessary Skips: 1
done in 0m0.89s
done (passed successfully)

Notice that we now have an “unnecessary skip”, which is true because the previously ignored error is no longer an error. We can safely remove the new line we added in .dialyzer_ignore.exs without worry. If we had left that line in the ignore file, we could have accidentally reintroduced the error without causing mix dialyzer to fail. Dialyxir gives us the ability to prevent these unnecessary skips, forcing us to remove fixed errors from the .dialyzer_ignore.exs file in order to pass type checks.

    # mix.exs
def project do
  [
    # ...
    dialyzer: [
      list_unused_filters: true
    ],
    # ...
  ]
end

  

Now when we have an unnecessary skip, mix dialyzer gives us the following error:

Total errors: 173, Skipped: 173, Unnecessary Skips: 1
Unused filters:
{"lib/phoenix.ex", :invalid_contract}
unused filters present
Halting VM with exit status 1

This is a slick way to prevent previously-fixed errors from sneaking back!

What Next?

Configuring CI

The next logical step is adding a Dialyzer check to our CI so that any PRs with new type errors are flagged and fixed before merging. The main concern is making sure that PLTs are cached properly between CI runs.

There is a section of the Dialyxir docs that explains how to configure various CI tools to work well with Dialyxir. And there’s the Fly GitHub Actions for Elixir CI/CD guide as well. We won’t go into too much other detail here.

Once we have Dialyzer integrated into our CI pipelines, we can prevent new type errors from showing up as well as starting to chip away at our .dialyzer_ignore.exs file by fixing existing errors!

When In Doubt, Ignore The Error

We’ve talked about how to ignore existing Dialyzer errors and prevent new ones from being added to a project, but it’s worth mentioning that ignoring new errors is completely valid as well. Sometimes a new error shows up (or you are trying to fix an existing ignored error) that neither you nor your colleagues can figure out. It might involve a massive nested struct or a series of complex function calls. The error message might span multiple pages of your terminal. Regardless, the purpose of a tool like Dialyzer is to improve developer experience/efficiency. If a tool starts to become more painful than helpful, escape hatches like Dialyzer’s .dialyzer_ignore.exs file are perfectly valid options. At the very least, adding a new error to your ignore file is a form of documentation of tech debt, not a failure on you or your colleagues’ part.

Types cannot catch every single bug or error; tests (unit, integration, or otherwise) are important as well, though they aren’t perfect either. We can raise our confidence in our code by combining types and tests; our goal should be productivity, not perfection!

What About Incremental Mode?

In 2022, Thomas Davies gave a talk titled “Incremental Dialyzer: How we made Dialyzer 7x Faster”. This new incremental mode for Dialyzer was released with OTP 26, and should drastically simplify and speed-up the addition of Dialyzer to large existing projects. It’s still super new, so few people (especially in the Elixir community) have had a chance to experiment with it. There is a GitHub issue about supporting this new --incremental mode in Dialyxir, but the work hasn’t been started yet.

This is an exciting enhancement that we have to look forward to!

Recommended Dialyzer Resources

You’ll notice that this post doesn’t cover “how to understand/fix Dialyzer errors”, as that is an entire topic worthy of its own post. Luckily, many people have written blog posts and given talks about this very topic. Here a collection of resources:

Aside: Language Servers

Up until now, all of our talk of type checking tools like Diayzer has involved running commands in our terminal, but many other programming languages often have tooling that will allow developers to interact with types and type-related errors in their favorite IDE or text editor. This is done using language servers. We won’t go too into depth, but a language server allows modern editors to hook into programming languages in order to provide things like intellisense, autocompletion, and type checking. These language servers are usually accompanied by an editor-specific extension to make use of the information they provide. Luckily, Erlang and Elixir both have solid options for language servers and extensions.

Erlang has Erlang LS, and many Elixir developers are familiar with ElixirLS. Both of these tools run Dialyzer in an incremental way. ElixirLS uses both Dialyzer and Dialyxir, but the latter is only used for formatting Dialyzer errors in a more readable fashion. ElixirLS stores its project PLT in .elixir_ls/dialyzer_manifest_$ERLANG_VERSION_elixir-$ELIXIR_VERSION_test file (without a .plt extension); if you ever run into issues with ElixirLS type errors not updating correctly, you can try to delete the PLT.

It’s also worth noting that alternative Elixir language servers exist, most notably Lexical and NextLS. Both Lexical and NextLS will support Dialyzer type-checking in the future; we look forward to seeing these tools develop!

Conclusion

We’ve seen how we can add Dialyzer late in the game to an Elixir project without getting overwhelmed. Tools like Dialyxir let us ignore all the legacy issues in our project today while helping us to keep it clean going forward. Hopefully we can keep in mind that our goal is for productivity, not perfection, and that it’s okay to ignore a type error so we can move forward today. We also touched on some interesting projects that may further improve our developer experience in future.

Finally, there is an ongoing research project by Giuseppe Castagna and Guillaume Duboc (and accompanying talk from ElixirConf EU 2023 by the project authors) looking to add a gradual type system to Elixir. That may change everything! Until that time, Dialyzer/Dialyxir can do a lot for our projects today. Hopefully we also saw that adding Dialyzer doesn’t have to be scary either.

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Acknowledgements

Everyone who participated in both of the mentioned Elixir Forum polls.
Jeremy Huffman for maintaining Dialyxir, being very accommodating to my various issues/PRs, and reviewing this blog post.
Jesper Eskilson for his talk, which was the catalyst in my Elixir type-checking quest.

Speed up your boot times with this one Dockerfile trick

2023-06-05T00:00:00+00:00

In this post, we look at optimizing boot up time for Machine Learning and Single File ElixirScripts. Fly.io is a great place to run your Elixir Applications! Check out how to get started!

Problem

You added Whisper text to speech transcription to your voice chat app and while it works great… the boot times on your deployment have slowed to a crawl. Or maybe you really love deploying your apps using single file elixir scripts but are finding the initial boot to be a little slower than you’d think. The common thread here is that they both will download and compile some dependencies at startup for you, what if we could do that in the build step?

Solution

The solution is actually pretty straightforward! In our Dockerfile we simply need to add a step where it runs this code and then stops. This will cause the files to be cached and available instantly at boot time. So let’s start with the Single File Elixir Scripts example:

    Mix.install([
  :req,
  {:jason, "~> 1.0"}
])

# Dry run for copying cached mix install from builder to runner
if System.get_env("EXS_DRY_RUN") == "true" do
  System.halt(0)
  Process.sleep(:infinity)
else
    Req.get!("https://api.github.com/repos/elixir-lang/elixir").body["description"]
    |> dbg()
end

  

Here we added an environment check for “EXSDRYRUN”, and if so we halt, otherwise we do the thing we wanted to spin up a docker container for in the first place!

Now lets see what changes we need to make to our Dockerfile.

    # syntax = docker/dockerfile:1
FROM "hexpm/elixir:1.14.2-erlang-25.2-debian-bullseye-20221004-slim"

# install dependencies
RUN apt-get update -y && apt-get install -y build-essential git libstdc++6 openssl libncurses5 locales \
    && apt-get clean && rm -f /var/lib/apt/lists/*_*

WORKDIR "/app"

# Copy our files over
COPY run.exs /app

# install hex + rebar if you plan on using Mix.install
RUN mix local.hex --force && \
    mix local.rebar --force

# force elixir to install your deps
ENV EXS_DRY_RUN=true
RUN elixir /app/run.exs

ENV EXS_DRY_RUN=false
CMD elixir /app/run.exs

  

What we’ve done here is run execute our code with EXS_DRY_RUN set to true during the docker build step, this will cache our dependencies in the docker container it’s self.

Just a couple extra lines and we’re good to go!

Bumblebee

Time for a more complex example using LiveBeats. Chris McCord semi-recently pushed an update that will transcribe your audio using bumblebee and the whisper model in real time. To set that up he has a function called load_serving that will configure, download and set up the model for you! Here is the code:

    def load_serving do
    {:ok, whisper} = Bumblebee.load_model({:hf, "openai/whisper-tiny"})
    {:ok, featurizer} = Bumblebee.load_featurizer({:hf, "openai/whisper-tiny"})
    {:ok, tokenizer} = Bumblebee.load_tokenizer({:hf, "openai/whisper-tiny"})

    Bumblebee.Audio.speech_to_text(whisper, featurizer, tokenizer,
        compile: [batch_size: 10],
        max_new_tokens: 100,
        defn_options: [compiler: EXLA]
    )
end

  

He calls this during application start and the deploys take a super long time because the application won’t fully boot till this is ready. The models are being downloaded from HuggingFace directly and then compiled into EXLA on start. To fix this, we simply call this function in our Dockerfile and it will cache the model , so it won’t download on boot. The one downside about this is that our docker image will get significantly larger depending on the size of the model.

Below is the full Dockerfile from LiveBeats:

    ARG BUILDER_IMAGE="hexpm/elixir:1.12.0-erlang-24.0.1-debian-bullseye-20210902-slim"
ARG RUNNER_IMAGE="debian:bullseye-20210902-slim"

FROM ${BUILDER_IMAGE} as builder

# install build dependencies
RUN apt-get update -y && apt-get install -y build-essential git \
    && apt-get clean && rm -f /var/lib/apt/lists/*_*

# prepare build dir
WORKDIR /app

# install hex + rebar
RUN mix local.hex --force && \
    mix local.rebar --force

# set build ENV
ENV MIX_ENV="prod"
ENV BUMBLEBEE_CACHE_DIR=/app/.bumblebee

# install mix dependencies
COPY mix.exs mix.lock ./
RUN mix deps.get --only $MIX_ENV
RUN mkdir config

# copy compile-time config files before we compile dependencies
# to ensure any relevant config change will trigger the dependencies
# to be re-compiled.
COPY config/config.exs config/${MIX_ENV}.exs config/
RUN mix deps.compile

COPY priv priv

# Compile the release
COPY lib lib

# note: if your project uses a tool like https://purgecss.com/,
# which customizes asset compilation based on what it finds in
# your Elixir templates, you will need to move the asset compilation
# step down so that `lib` is available.
COPY assets assets

# compile assets
RUN mix assets.deploy

RUN mix compile

# Changes to config/runtime.exs don't require recompiling the code
COPY config/runtime.exs config/

# NEW HERE
# Download the HuggingFace models to cache them
RUN /app/bin/live_beats eval 'LiveBeats.Application.load_serving()'

COPY rel rel
RUN mix release

# start a new build stage so that the final image will only contain
# the compiled release and other runtime necessities
FROM ${RUNNER_IMAGE}

RUN apt-get update -y && apt-get install -y libstdc++6 openssl libncurses5 locales \
  && apt-get clean && rm -f /var/lib/apt/lists/*_*

# Set the locale
RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && locale-gen

ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8


WORKDIR "/app"
RUN chown nobody /app

# Only copy the final release from the build stage
COPY --from=builder --chown=nobody:root /app/.bumblebee/ ./.bumblebee
COPY --from=builder --chown=nobody:root /app/_build/prod/rel/live_beats ./

USER nobody

# Set the runtime ENV
ENV ECTO_IPV6="true"
ENV ERL_AFLAGS="-proto_dist inet6_tcp"
# NEW HERE
ENV BUMBLEBEE_CACHE_DIR=/app/.bumblebee
ENV BUMBLEBEE_OFFLINE=true

CMD /app/bin/server

  

The magic lines are highlighted with # NEW HERE mainly during the build stage we setup an environment variable for where Bumblebee should put our cached models, execute LiveBeats.Application.load_serving(),this will run our code without booting up the entire application, copy the those files into our runner and set ENV BUMBLEBEE_OFFLINE=true which makes sure that our app will still run even if HuggingFace goes down using the cached models.

And with that single line, we’ve done it! Now you have your cache built into your docker container!

Discussion

We showed two examples moving a boot-time step to a build-time step. This can be exceptionally handy when we deploy to many regions or want a fast loading application. In the case of BumbleBee, we also remove a risk that we won’t be able to boot if HuggingFace is not reachable.

Try to identify work that is happening on boot, and if its cached to the file system. If it is, move that work to the build step if possible!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Migrating to Verified Routes

2023-06-05T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. This post is about how to migrate from legacy Phoenix routes to the new Verified Routes in Phoenix 1.7. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Verified Routes were introduced in Phoenix 1.7. The upgrade guide strongly suggests upgrading to Phoenix 1.7 first and keeping your routes the same to begin with. Then later, you can migrate the routes as needed.

Well, it’s later. Now let’s figure out how to migrate those routes!

Problem

While the application is working fine as-is, you want the benefits of Verified Routes. Namely, those are:

Easier to read
Easier to write
Shorter and more elegant: No route helper function with a bunch of arguments
Compile-time checked: Even though they look like a normal string

You have an Elixir Phoenix project that was upgraded to Phoenix 1.7+. A lot of code was written before Verified Routes were available and now you’d like to migrate over to use them. The project is large and there are a lot of routes to convert and files to update. How can we best migrate to use Verified Routes? What can we expect from the process?

Solution

I went through the migration process with a project and took notes along the way. I’ll share what worked, what didn’t, what I did, and what I learned.

There is a slick tool to help with the conversion process. It’s a mix task created by Andreas Eriksson and can be found in this Gist. This tool was really helpful and does the bulk of the grunt work.

However, before we create and run the mix task, we’ll do a check and perform a manual conversion if needed. Let’s take a peek at what’s ahead.

A quick, manual pre-migration stop - But only if needed.
- Static paths
Tool: mix convert_to_verified_routes to migrate the bulk of it.
What doesn’t get migrated - and how we deal with it.
- Multi-line routes
- Some query params
Updating components that use verified routes
Finishing up

TIP: Before starting, make sure you have a clean git workspace so you can easily review and revert changes if needed.

Before we dive in and start the migration, we’ll do a quick check and manual migrate any static routes.

A Manual Static Path Conversion

The mix task we’ll use breaks if it encounters any static routes. Thankfully, they are easy to find and convert.

Do a search for “Routes.static_” to see if any exist in the project. If not, move right along

Found a static route?

Static paths break things

If you have any static path routes that use Routes.static_path/2 like the following, they need to be converted manually.

    Routes.static_path(@socket, "/images/logo-100px.png")

  

These completely confuse the script. They break the conversion process and it won’t detect more routes once this is encountered. They need to be handled manually. Thankfully, there are probably very few of them and the conversion is really straightforward.

Convert from:

    Routes.static_path(@socket, "/images/logo-100px.png")

  

To:

~p"/images/logo-100px.png"

At least it’s easy!

A Tool to Help Migrate

The tool is a mix task found in this Gist. The mix task has a dedicated blog post: Automatic conversion to verified routes and it looks something like this when run:

With any static routes dealt with, we’re ready to start the migration! Let’s see how far we can get using the mix task.

Create the Mix Task

Create the file my_app/lib/mix/tasks/convert_to_verified_routes.ex and copy the contents of the Gist into it. For reference, here’s what the Elixir School has to say about Custom Mix Tasks.

Next, update the @web_module to match the one in the application being migrated.

@web_module MyAppWeb

Before we can run the mix task, we need to compile our project so the mix task becomes available: mix compile

We’re ready to run the script!

Running the Mix Task

With the mix task ready, let’s start converting! 🤞

We execute the mix task like this:

mix convert_to_verified_routes

What does it do?

When we run the mix task, it does a regex search through the source code of the project and finds instances like Routes.my_route_path(socket, :index) and prompts to convert them to the appropriate verified route.

The prompt looks something like this:

The capital "Y" at the end of the prompt means it’s the default action and we can just hit ENTER to accept the change.

Should we replace
  Routes.shop_specials_index_path(socket, :index)
with
  ~p"/shop/specials"
 [Yn]

The task searches through the full project, including test files. 🎉 It does a pretty decent job of converting most of your routes. It even detects and substitutes most query parameters!

What Doesn’t Get Migrated?

The mix task does not do 100% of the conversion. For instance:

It doesn’t convert multi-line routes.
It doesn’t handle some query parameters correctly.
Routes in comments are not converted. This is more of a “heads-up” than an issue.
If any routes are invalid, they are not updated and replaced. This makes sense, but there are no warnings of this either. You may first suspect that it didn’t work and then find out the route was bad all along!

To find routes that weren’t converted, search the project for Routes.. Also, depending on the project, the conversion may result in some non-compiling code. On the plus side, that helps find where is didn’t work. This was the case for some of my query parameter conversions.

First, let’s come back to the issue with multi-line routes.

Multi-Line Routes

Long routes get wrapped by the Elixir formatter. These don’t get updated. For example:

    <.link
  patch={
    Routes.admin_blog_section_show_path(
      @socket,
      :confirm_delete,
      @blog,
      @article
    )
  }
  class="..."
>

  

The route was wrapped by the code formatter and crosses multiple lines. This route will not be converted automatically. To fix this, we can manually reformat the text up onto a single line and re-run the mix task. Lo and behold, it now gets converted!

Query Param Issues

There are a number of different ways to pass params to a route. Depending on what’s in the project, it may or may not have an issue.

In my project, I had some query params that were passed as a string. The URL query might look like this:

http://localhost:4000/partner/integration?by=enabled

When attempting to replace the params, this is what the conversion prompt looked like:

Should we replace
  Routes.partner_integration_index_path(socket, :index, by: "enabled")
with
  ~p"/partner/integration?#{[by: \enabled\]}"
 [Yn]

It didn’t handle the "enabled" string literal correctly and it output invalid code. My usage was for sort order, so it was easy to search for [by: \ and find them all quickly. It helped seeing what it converted them to. Alternatively, we could say “no” to replace and handle it manually.

At the time, I looked into updating the script to handle this use-case, but I only had like 6 places where it was used. Since this was a one-time, one-way conversion, I was fine just doing the manual clean-up.

Components With Routes in Them

Separate from the migration process, there was another change I needed to make when moving to verified routes.

I had some components, like those for a sidebar menu, that had routes inside the component. Here’s an example. Notice that “target” takes a verified route and it’s inside the profile_sidebar function component.

      attr :active, :atom, required: true

  def profile_sidebar(assigns) do
    ~H"""
    <nav class="space-y-1">
      <.sidebar_nav_item
        title="Account"
        name={:account}
        icon="far fa-user"
        active={@active}
        target={~p"/account"} />
      ...
    </nav>
    """
  end

  

It showed up as the error: warning: undefined function sigil_p/2.

The fix was to add the following line to the top of my component file.

    use Phoenix.VerifiedRoutes,
  endpoint: MyAppWeb.Endpoint,
  router: MyAppWeb.Router

  

With that addition, I could use Verified Routes in a component. Yay!

Finishing Up

The mix task did the bulk of the work for me and converted 80 files. There were multiple passes of running the script, finding something that wasn’t converted, updating the project and running the script again.

After the migration was complete, there were no more Routes. found in the project. My components were updated when using verified routes internally, the project compiled, and the tests pass. Done! 😅

The final cleanup was to delete the mix task from the project. No need to keep it around.

Discussion

There is no silver bullet to convert everything perfectly in one shot. Still, the mix task gets us really far! When we know what kinds of left-overs might exist and how to find them, we can pretty quickly get a project migrated over to use Verified Routes.

Once migrated, we get all of the lovely benefits of Verified Routes. Here’s a refresher on why Verified Routes are worth the effort.

Much easier to write
Much easier to read
Shorter and more elegant
Great compiler checks

Thanks to Andreas Eriksson for putting in the effort to make a tool like this available to the community!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Dynamic forms with LiveView Streams

2023-05-29T00:00:00+00:00

In this post, we’ll develop a dynamic list component using the new LiveView Streams and the enhanced form features introduced in LiveView 0.18/0.19. Get ready to discover how to create a responsive list component with interactive capabilities like adding, editing, and deleting items. Fly.io is a great place to run your Phoenix LiveView applications! Check out how to get started!

You developed a component that lets you add and remove items in a “has_many” relationship. Take, for example, a shopping list where it can have many items. You’ve got a cool form that lets you send data for new items, and it works really well:

However, you’re still not entirely happy with the result. When you make a mistake while adding an item to the list, there’s no way to edit it. How can you make the list items editable right in the list component, without the need for a modal?

Well, what if we use individual forms for each list item instead of a single form to add and associate them with the list? That’s a simple task, we can render as many forms as we want. We’re going to use the to_form/2 function to create form structs from changesets, and then we’re going to use those structs to render our form components.

And another thing: once we’ve fixed up our form component, we’re going to see how to use LiveView Streams to manage and manipulate our list items without the need to store them all in memory. Adding, editing, deleting, and resetting items in a collection has never been easier!

Defining our component’s markup

Let’s begin by refactoring our component and keeping only the header:

    defmodule ComponentsExamplesWeb.ListComponent do
  use ComponentsExamplesWeb, :live_component

  def render(assigns) do
    ~H"""
    <div class="bg-gray-100 py-4 rounded-lg">
      <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
        <.header>
          <%= @list_name %>
        </.header>
      </div>
    </div>
    """
  end

  def update(%{list: list} = assigns, socket) do
    socket =
      socket
      |> assign(
        list_id: list.id,
        list_name: list.title
      )
    {:ok, socket}
  end
end  

  

In the assigns, we’re passing a struct list. It contains an id, a title, and a list of items, each of which has multiple attributes.

To generate these structs, their corresponding changesets, and the context functions that we’ll be using later, you can use the phx.gen.context generator:

mix phx.gen.context SortableList List lists title:string
mix phx.gen.context SortableList Item items name:string position:integer list_id:integer

Here’s an example list, containing a single item:

    %ComponentsExamples.SortableList.List{
  __meta__: #Ecto.Schema.Metadata<:loaded, "lists">,
  id: 1,
  title: "Shopping list 1",
  items: [
    %ComponentsExamples.SortableList.Item{
      __meta__: #Ecto.Schema.Metadata<:loaded, "items">,
      id: 161,
      name: "chocolate",
      position: 0,
      status: :started,
      list_id: 1,
      list: #Ecto.Association.NotLoaded<association :list is not loaded>,
      inserted_at: ~N[2023-05-16 20:29:12],
      updated_at: ~N[2023-05-16 20:29:12]
    }
  ],
  inserted_at: ~N[2023-04-25 19:35:09],
  updated_at: ~N[2023-04-25 19:35:09]
}

  

Now, we need to iterate over the items of the list and render the forms. But before we dive into that, let’s update our assigns to include a form per item:

    def update(%{list: list} = assigns, socket) do
+ item_forms = Enum.map(list.items, &build_item_form(&1, %{list_id: list.id}))

  socket =
    socket
    |> assign(
      list_id: list.id,
      list_name: list.title,
+     items: item_forms
    )
  {:ok, socket}
end

  

Let’s see the content of the build_item_form/1 function:

    defp build_item_form(item_or_changeset, params) do
  changeset =
    item_or_changeset
    |> SortableList.change_item(params)

  to_form(changeset, id: "form-#{changeset.data.list_id}-#{changeset.data.id}")
end

  

We receive an %Item{} and create a changeset from it. Then, we use the Phoenix.Component.html.to_form/2 function. This handy function converts a data structure into a Phoenix.HTML.Form, which we can use to render our form/1 components. Note that the id of the form has the list_id and the id of the item interpolated.

Check out how to_form/2 allows change tracking on individual inputs in our forms!

As a result, we have an assign called :items that holds the Phoenix.HTML.Form structs containing our element data. We iterate through each item in our assign and render a simple_form/1 for each of them:

You can find the simple_form/1 component—and many others—in core_component.ex

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
      <.header>
        <%= @list_name %>
      </.header>
+     <div id={"#{@list_id}-items"} class="grid grid-cols-1 gap-2">
+      <div :for={form <- @items} id={"list#{@list_id}-item#{item.id}"}>
+       <.simple_form
+           for={form}
+           phx-change="validate"
+           phx-submit="save"
+           phx-target={@myself}
+           class="min-w-0 flex-1 drag-ghost:opacity-0"
+           phx-value-id={form.data.id}
+         >
+          <div class="flex">
+          </div>
+         </.simple_form>
        </div>
      </div>
    </div>
  </div>
  """
end

  

We define the events triggered on change or data submission, and we pass the item ID to these events using the phx-value-* bindings.

Tip: Make sure you’re using LiveView 0.19 or a newer version, as it was not possible to send phx-value-* values with forms before this update.

Now, let’s add elements to the body of our item form. We start by adding a button to change the status of our element:

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
      <.header>...</.header>
      <div id={"#{@list_id}-items"} class="grid grid-cols-1 gap-2">
        <div :for={form <- @items} id={"list#{@list_id}-item#{item.id}"}>
          <.simple_form ...>
            <div class="flex">
+.            <button
+               :if={form.data.id}
+               type="button"
+               class="w-10"
+.              phx-click={JS.push("toggle_complete", target: @myself, value: %{id: form.data.id})}
+             >
+               <.icon
+                 name="hero-check-circle"
+                 class={[
+                   "w-7 h-7",
+.                  if(form[:status].value == :completed, 
+                     do: "bg-green-600", 
+                     else: "bg-gray-300")
+                 ]}
+               />
+.            </button>
            </div>
          </.simple_form>
        </div>
      </div>
    </div>
  </div>
  """
end

  

This button is displayed conditionally when the list element has an id, indicating that it already exists in the database and its status can be edited. Additionally, we apply conditional classes to the <.icon> based on the item’s status, to change its color.

Now, let’s add the most important part, the text input to to edit each list item:

We add two text inputs to send the parameters of our item: the :name and the :list_id to which it belongs:

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
      <.header>...</.header>
      <div id={"#{@list_id}-items"} class="grid grid-cols-1 gap-2">
        <div :for={form <- @items} id={"list#{@list_id}-item#{item.id}"}>
          <.simple_form ...>
            <div class="flex">
              ...
+             <div class="flex-auto block text-sm leading-6 text-zinc-900">
+               <input type="hidden" name={form[:list_id].name} value={form[:list_id].value} />
+               <.input
+                 field={form[:name]}
+                 type="text"
+                 phx-target={@myself}
+                 phx-key="escape"
+                 phx-keydown={
+                   !form.data.id &&
+                     JS.push("discard", target: @myself, value: %{list_id: @list_id})
+                 }
+                 phx-blur={form.data.id && JS.dispatch("submit", to: "##{form.id}")}
+               />
+             </div>
            </div>
          </.simple_form>
        </div>
      </div>
    </div>
  </div>
  """
end

  

Let’s take a closer look at the elements we added. First, we include a hidden input field to store the id of the list to which the element belongs. This allows us to send it as part of the parameters for the validate and save events by setting the value={form[:list_id].value} attribute.

Next, we introduce a slightly more complex <.input> component with event options. By using the phx-key and phx-keydown attributes, we specify that pressing the escape key triggers the discard event sent to the server.

We have two ways to save changes: pressing Enter to trigger the submit event or allowing changes to be automatically saved when the input loses focus.

For elements that already exist in the database and are modified, the phx-blur binding comes into play. It automatically submits the form when the input loses focus, ensuring that changes are saved seamlessly.

Lastly, we add a button with an icon to delete existing elements from the database:

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
      <.header>...</.header>
      <div id={"#{@list_id}-items"} class="grid grid-cols-1 gap-2">
        <div :for={form <- @items} id={"list#{@list_id}-item#{item.id}"}>
          <.simple_form ...>
            <div class="flex">
              ...
+             <button
+               :if={form.data.id}
+               type="button"
+               class="w-10 -mt-1 flex-none"
+               phx-click={
+                 JS.push("delete", target: @myself, value: %{id: form.data.id})
+                 |> hide("#list#{@list_id}-item#{form.data.id}")
+               }
+             >
+               <.icon name="hero-x-mark" />
+             </button>
            </div>
          </.simple_form>
        </div>
      </div>
    </div>
  </div>
  """
end

  

We trigger the delete event while simultaneously hiding the form of the item we wish to remove.

We now add the final section to our component. We include two buttons: one to add a new item to the list, and another to delete all items from the list.

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
      <.header>...</.header>
      <div id={"#{@list_id}-items"} class="grid grid-cols-1 gap-2">
        <div :for={form <- @items} id={"list#{@list_id}-item#{item.id}"}>
          ...
        </div>
      </div>
+     <.button phx-click={JS.push("new", target: @myself, value: %{list_id: @list_id})} class="mt-4">
+       Add item
+     </.button>
+     <.button
+       phx-click={JS.push("reset", target: @myself, value: %{list_id: @list_id})}
+       class="mt-4"
+     >
+       Reset
+     </.button>
    </div>
  </div>
  """
end

  

We sent events to the server using JS.push. Both the reset and new events receive the list_id as a parameter.

Awesome! We’ve finished the markup for our component!

We can easily render it by passing a list struct like the one we saw above:

    def render(assigns) do
  ~H"""
  <div id="lists" class="grid sm:grid-cols-1 md:grid-cols-3 gap-2">
    <.live_component
      :for={list <- @lists}
      module={ComponentsExamplesWeb.ListComponent}
      id={list.id}
      list={list}
    />
  </div>
  """
end

  

But before we tackle handling all the events we defined, how about we optimize memory usage by using Streams? With just a few tweaks, we can implement this feature and ensure we’re not storing all the list elements in memory.

Converting to Streams

To optimize memory usage, let’s start by making a small change to our assigns. We assign the stream items using the stream/4 function:

    def update(%{list: list} = assigns, socket) do
  item_forms = Enum.map(list.items, &build_item_form(&1, %{list_id: assigns.id}))

  socket =
    socket
    |> assign(
      list_id: list.id,
      list_name: list.title,
-     items: item_forms
    )
+   |> stream(:items, item_forms)

  {:ok, socket}
end

  

Next, we define the required phx-update DOM attribute on the parent container where the item collection is rendered. The items are now accessed via the new assign @streams, and we consume the stream using a comprehension:

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
      ...
-      <div id={"#{@list_id}-items"} class="grid grid-cols-1 gap-2">
+      <div
+        id={"#{@list_id}-items"}
+        class="grid grid-cols-1 gap-2"
+        phx-update="stream"
+      >
-        <div :for={form <- @items} id={"list#{@list_id}-item#{item.id}"}>
+        <div :for={{id, form} <- @streams.items} id={id}>         
        ...
        </div>
      </div>
      ...
    </div>
  </div>
  """
end

  

Now we’re all set to handle the events we defined earlier!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Add, delete, update, reset Streams

We’ll be defining five events (new, validate, save, delete, reset). Our main focus is explain how we can use streams to reflect changes in our list. We won’t be diving into functions that interact with the database.

To get started, it will be helpful to define a helper function that creates forms already associated with the list_id:

    defp build_empty_form(list_id) do
  build_item_form(%Item{list_id: list_id}, %{})
end

  

Keep in mind this function, as well as the one we defined earlier. We’ll be using them in our upcoming steps!

New

To add a new element to the list, we can use the function we just defined to create an empty form. Then, we can insert this form into the stream using the stream_insert/4 function:

    def handle_event("new", %{"list_id" => list_id}, socket) do
  {:noreply, stream_insert(socket, :items, build_empty_form(list_id), at: -1)}
end

  

To insert the new form at the end of the list, we use the :at option provided by the stream_insert/4 function.

Validate

To display the errors of our item, we need to modify the already rendered form in the client and insert a new form that includes the errors from the changeset.

One important detail to note is that in order for our component to recognize that it should display the changeset errors, we need to add an :action to it. To achieve this, we make a slight modification to the build_item_form/4 function that we defined earlier:

    -defp build_item_form(item_or_changeset, params) do
+defp build_item_form(item_or_changeset, params, action \\ nil) do
  changeset =
    item_or_changeset
    |> SortableList.change_item(params)
+   |> Map.put(:action, action)

  to_form(changeset, id: "form-#{changeset.data.list_id}-#{changeset.data.id}")
end

  

The action can be any random atom, but hey, let’s keep things clear and name them sensibly.

Now let’s see how to handle the event and use this new :action parameter:

    def handle_event("validate", %{"item" => item_params} = params, socket) do
  item = %Item{id: params["id"] || nil, list_id: item_params["list_id"]}
  item_form = build_item_form(item, item_params, :validate))
  {:noreply, stream_insert(socket, :items, item_form}
end 

  

First, we generate a new %Item{} that includes the item id sent from the text input and the list_id sent using the phx-value-id={form.data.id} parameter. Next, we call our build_item_form/4 function with the :validate action and insert the item into the stream using stream_insert/4.

Like magic, we didn’t have to specify that this is an update of a form that already exists! This is because to_form created a DOM ID that allows to identify the elements of the stream that already exists in the client.

Save

When attempting to save a new item, we may receive one of two possible responses. First, when an item is successfully inserted, we clear the new item form and replace it with a fresh empty form. We also insert a new form containing the persisted data. Second, if an error occurs, we display the relevant errors to the user.

Let’s now delve into the implementation details of these actions.

    def handle_event("save", %{"item" => item_params}, socket) do
  case SortableList.create_item(item_params) do
    {:ok, new_item} ->
      empty_form = build_empty_form(item_params["list_id"])


      {:noreply,
       socket
       |> stream_insert(:items, build_item_form(new_item, %{}))
       |> stream_delete(:items, empty_form)
       |> stream_insert(:items, empty_form)}

    {:error, changeset} ->
      {:noreply, assign(socket, :form, build_item_form(changeset, %{}, :insert)))}
  end
end

  

Great! We can chain together the different functions of the stream using the pipeline operator.

On the other hand, the save event may be triggered by one of the forms that have already been saved in the database, indicating an update of the item. We can identify this by pattern matching, receiving a non-null item_id:

    def handle_event("save", %{"id" => item_id, "item" => params}, socket) do
  todo = SortableList.get_item!(item_id)

  case SortableList.update_item(todo, params) do
    {:ok, updated_item} ->
      {:noreply, stream_insert(socket, :items, build_item_form(updated_item, %{}))}

    {:error, changeset} ->
      {:noreply, stream_insert(socket, :items, build_item_form(changeset, %{}, :update))}
  end
end

  

It’s almost like magic, isn’t it? We just need to use stream_insert, and the stream takes care of updating itself. And if we want to display the errors from the changeset, we simply add an :action to it.

Delete

Deleting items from a stream is easy too!

For this we have stream_delete/3 which also receives changesets as a parameter:

    def handle_event("delete", %{"id" => item_id}, socket) do
  item = SortableList.get_item!(item_id)
  {:ok, _} = SortableList.delete_item(item)
  {:noreply, stream_delete(socket, :items, build_item_form(item, %{}))}
end

  

You can also use the stream_delete/3 function to discard the form of the new element when needed:

    def handle_event("discard", params, socket) do
  {:noreply, stream_delete(socket, :items, build_empty_form(params["list_id"]))}
end

  

Reset

We have the option to remove all items from a stream, which is perfect for our reset button:

    def handle_event("reset", params, socket) do
  empty_form = build_empty_form(params["list_id"])
  {:noreply, stream(socket, :items, [empty_form], reset: true)}
end

  

We simply need to reconfigure our stream by passing the option reset: true. Additionally, we can include a list of new elements to be inserted, which in this case would be at least one empty form.

LiveView 0.19 added support for stream resets with bulk insert operations!

Hooray! We’ve done it! Our component is now complete and ready to shine!

Closing

The features we explored today unlock a world of exciting new possibilities for apps development with LiveView. LiveView Streams revolutionize collection handling and memory optimization, simplifying tasks such as adding, editing, and deleting items. Furthermore, the optimizations brought by to_form/1 enable efficient manipulation of individual inputs without the need for full form re-rendering. This simple yet immensely powerful function opens up new avenues for form usage, expanding the potential of your applications.

Check out this repo to see these game-changing features in action. We used our previous learnings to create an even more impressive component!

Credits

A big shoutout to Chris McCord for sharing the incredible example that inspired these posts and for patiently answering any questions about the exciting new concepts in Phoenix and LiveView.

Taking Control of Map Sort Order in Elixir

2023-05-18T00:00:00+00:00

This post is about customizing IEx to sort map keys by default after a change in OTP 26. If you are looking for place to launch your latest Elixir project, check out how to get started on Fly.io! You could be up and running in minutes.

If you’ve recently upgraded to Elixir 1.14.4 and OTP 26, you may have noticed that map keys are no longer showing in a predictable order. Fear not! Understanding why it’s happening and taking the time to sort the keys by default can save you some headaches. In addition, we can explore a few other neat tricks with customizing IEx.

Problem

You recently upgraded your OTP to 26 or a later version. To your surprise, the maps you inspect now display with their keys in seemingly random order. In IEx, it can be frustrating to find things when map key order has no rhyme or reason. For example:

    data = %{a: 1, b: 2, c: 3, d: 4}
%{c: 3, a: 1, d: 4, b: 2}

Did you notice how the keys in the output version are arranged?

How can we predictably sort the map keys when working in IEx?

Solution

Why this is happening

The recent internal changes in OTP 26 is the culprit. As stated in the Erlang OTP Release Notes:

Some map operations have been optimized by changing the internal sort order of atom keys. This changes the (undocumented) order of how atom keys in small maps are printed and returned. The new order is unpredictable and may change between different invocations of the Erlang VM.

The Elixir 1.14.4 release notes also calls this out:

When migrating to Erlang/OTP 26, keep it mind it changes how maps are stored internally and they will be printed and traversed in a different order (note maps never provided a guarantee of their order). To aid migration, this release adds :sort_maps to inspect custom options, in case you want to sort them before inspection.

Let’s see what that :sort_maps option is and how it’s used.

    iex> data = %{a: 1, b: 2, c: 3, d: 4}
iex> IO.inspect(data, custom_options: [sort_maps: true])
%{a: 1, b: 2, c: 3, d: 4}

  

When we call inspect explicitly, there is a new custom_options of sort_maps: true. That can be helpful when logging a map in production, but it’s a lot to type when we’re developing locally.

Let’s see if we can make the local experience better.

Sorting by Default

When working in IEx locally, we want it sorted by default. This can be achieved by configuring the .iex.exs file. It can be located in your home directory or with your project. If you don’t have one yet, create it. Add the following code:

    IEx.configure(
  inspect: [
    custom_options: [sort_maps: true]
  ]
)

  

Test it out in a new IEx session:

    iex> data = %{a: 1, b: 2, c: 3, d: 4}
%{a: 1, b: 2, c: 3, d: 4}

Nice! Sorted by default!

What else can we do in our `.iex.exs` file?

Over time I’ve picked up some IEx customizations from others. Here’s my current .iex.exs file:

    Application.put_env(:elixir, :ansi_enabled, true)

timestamp = fn ->
  {_date, {hour, minute, _second}} = :calendar.local_time
  [hour, minute]
  |> Enum.map(&(String.pad_leading(Integer.to_string(&1), 2, "0")))
  |> Enum.join(":")
end

IEx.configure(
  colors: [
    syntax_colors: [
      number: :light_yellow,
      atom: :light_cyan,
      string: :light_black,
      boolean: :red,
      nil: [:magenta, :bright],
    ],
    ls_directory: :cyan,
    ls_device: :yellow,
    doc_code: :green,
    doc_inline_code: :magenta,
    doc_headings: [:cyan, :underline],
    doc_title: [:cyan, :bright, :underline],
  ],
  default_prompt:
    "#{IO.ANSI.green}%prefix#{IO.ANSI.reset} " <>
    "[#{IO.ANSI.magenta}#{timestamp.()}#{IO.ANSI.reset} " <>
    ":: #{IO.ANSI.cyan}%counter#{IO.ANSI.reset}] >",
  alive_prompt:
    "#{IO.ANSI.green}%prefix#{IO.ANSI.reset} " <>
    "(#{IO.ANSI.yellow}%node#{IO.ANSI.reset}) " <>
    "[#{IO.ANSI.magenta}#{timestamp.()}#{IO.ANSI.reset} " <>
    ":: #{IO.ANSI.cyan}%counter#{IO.ANSI.reset}] >",
  history_size: 50,
  inspect: [
    pretty: true,
    limit: :infinity,
    width: 80,
    custom_options: [sort_maps: true]
  ],
  width: 80
)

  

Here’s how a default IEx looks:

Here’s how my customized one looks. It features improved syntax colors and a customized prompt.

There are other customizations in there. Feel free to adapt or adopt only what you care about.

Discussion

We’ve learned why the key order is no longer predictable, how to sort keys before inspection, and how to configure IEx to sort keys by default. We’ve also seen that there are more customization possibilities for our IEx shell. (Beware the rabbit hole!)

If this tip was helpful for you, be sure to share it with your team!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Loading a structure.sql file on Prod without mix

2023-05-17T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. This post is about how to deploy your Elixir app to a fresh database after you’ve run mix ecto.dump and deleted old migration files. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Problem

At some point, you or your team ran mix ecto.dump and generated a priv/repo/structure.sql file. Then old migrations files were removed. Things were great and you could handle the local dev and test setup using mix ecto.load. If you were migrating your database to a new server, there are other options available.

The problem we’re facing comes after building a mix release and we are deploying to an empty database in a production environment where mix is not available. How do we load our structure.sql file on the server when mix ecto.load is not available?

When might this happen?

Deploying a new clean staging environment.
Testing a mature app on Fly.io before bringing over the full database.

Solution

Assuming the mix ecto.dump command was used, we have a priv/repo/structure.sql file generated for us. That file should be committed to our version control system. When we run fly launch or fly deploy, by default the file will be included and that means it will be present on the server. That gets us part way there.

The problem we have in our production environment is that the development tool mix is not available on the server after we build and deploy a release. This means our mix ecto.setup solution doesn’t work in production.

We need to take extra steps to get our structure.sql loaded. We will rely on the Ecto.Adapter.Structure.structure_load/2 function to do a lot of the work, but it depends on the psql command being available on the server.

To run psql on the server, we need to add the postgresql-client package to our Dockerfile in the final layer. Let’s do that now.

    # ... build layer

# start a new build stage so that the final image will only contain
# the compiled release and other runtime necessities
FROM ${RUNNER_IMAGE}

RUN apt-get update -y && apt-get install -y libstdc++6 openssl libncurses5 locales postgresql-client \
  && apt-get clean && rm -f /var/lib/apt/lists/*_*

#...

We added postgresql-client to the end of our installed packages. These are the packages included in the final runtime layer.

NOTE: If the Postgres package is added to an earlier build layer, it will not be available on the server.

The big change happens in our release.ex file. When we run fly launch, it executes mix phx.gen.release for us and that generates a MyApp.Release module. This module defines the migrate/0 function that gets executed in our production environment.

Here’s an example of how to update the migrate/0 function and add a check_and_execute_structure_sql/1 function that runs the structure.sql file when needed:

      require Logger

  def migrate do
    load_app()

    for repo <- repos() do
      # Ensure the repo is started. Then, if the "schema_migrations"
      # table does not exist, run the `structure.sql` file for the repo.
      {:ok, _, _} = Ecto.Migrator.with_repo(repo, &check_and_execute_structure_sql(&1))

      {:ok, _, _} = Ecto.Migrator.with_repo(repo, &Ecto.Migrator.run(&1, :up, all: true))
    end
  end

  # ...

  defp check_and_execute_structure_sql(repo) do
    config = repo.config()
    app_name = Keyword.fetch!(config, :otp_app)

    if Ecto.Adapters.SQL.table_exists?(repo, "schema_migrations") do
      Logger.info("schema_migrations table already exists")
      :ok
    else
      case repo.__adapter__().structure_load(
             Application.app_dir(app_name, "priv/repo"),
             repo.config()
           ) do
        {:ok, location} = success ->
          Logger.info("The structure for #{inspect(repo)} has been loaded from #{location}")
          success

        {:error, term} when is_binary(term) ->
          Logger.error("The structure for #{inspect(repo)} couldn't be loaded: #{term}")
          {:error, inspect(term)}

        {:error, term} ->
          Logger.error("The structure for #{inspect(repo)} couldn't be loaded: #{inspect(term)}")
          {:error, inspect(term)}
      end
    end
  end

  

The needed code was taken and adapted from the mix ecto.load task in ecto_sql. Pulling the code out of the mix task was necessary because the dev tool mix isn’t available on the server after generating a release. Ideally, this code could be extracted and made more readily available upstream.

With these additional steps, we can ensure that our structure.sql file is loaded in both a local environment and on the server. With that taken care of, we can run mix ecto.dump, delete old migrations, and still support deploying to a fresh database when needed.

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Streaming OpenAI responses

2023-05-17T00:00:00+00:00

This post is about using Elixir to stream OpenAI Chat Response’s in real time! If you want to keep your latency low then check out how to get started. You could be up and running in minutes.

Problem

You are building an application that interfaces with OpenAI’s ChatGPT and want to create a real time interactive experience just like the OpenAI Chat UI.

To do this we will need to work with the ChatGPT Streaming API, which is built using the HTTP Server Sent Events . Elixir is great for real time applications, how can we use the streaming API with Elixir?

Solution

Server Sent Events are a streaming response protocol compatible with HTTP/1.1. A GET request is made to a server and will keep the connection alive sending messages in the format data: <message>\n\n until the connection closes. Browsers handle this by parsing the data line by line and giving you the message stream. If you are curious, yes Plug does support it!

Let’s start by adding the fantastic Req to your dependencies. Req is a high level HTTP Client Library built by Elixir Contributor Wojtek Mach. It builds off of pure Elixir libraries and uses common Elixir idioms and patterns. It also comes with tons of developer UX such has handlers for common response types, streaming requests, and common header values.

Overall if we want a “just works” http client library use Req, if we want something a little lower level use Finch, which is what Req is built on top of. Today we will end up using both!

        {:req, github: "wojtekmach/req"}

  

We’re using the main branch here until a version > 0.3.6 is deployed. The fine grained control of Streams was just added to Req and will be available with the next version. We could have just used the Finch Library directly but Req is handy enough I still grabbed it!

We’re going to make a single function called gpt_stream that takes a prompt and callback function. And luckily for us the Req library has an example from the documentation that handle’s this case! So building off of that:

    defmodule OpenAI do
  def gpt_stream(prompt, cb) do
    fun = fn request, finch_request, finch_name, finch_options ->
      fun = fn
        {:status, status}, response ->
          %{response | status: status}

        {:headers, headers}, response ->
          %{response | headers: headers}

        {:data, data}, response ->
          body =
            data
            |> String.split("data: ")
            |> Enum.map(fn str ->
              str
              |> String.trim()
              |> decode_body(cb)
            end)
            |> Enum.filter(fn d -> d != :ok end)

          old_body = if response.body == "", do: [], else: response.body

          %{response | body: old_body ++ body}
      end

      case Finch.stream(finch_request, finch_name, Req.Response.new(), fun, finch_options) do
        {:ok, response} -> {request, response}
        {:error, exception} -> {request, exception}
      end
    end

    Req.post!("https://api.openai.com/v1/chat/completions",
      json: %{
        # Pick your model here
        model: "gpt-3.5-turbo-0301",
        messages: [%{role: "user", content: prompt}],
        stream: true
      },
      auth: {:bearer, System.fetch_env!("OPENAI_KEY")},
      finch_request: fun
    )
  end

  defp decode_body("", _), do: :ok
  defp decode_body("[DONE]", _), do: :ok
  defp decode_body(json, cb), do: cb.(Jason.decode!(json))
end

  

Some functions are easier to read from the bottom up, so let’s start there. Req.post!()takes the usual parameters:

URL
JSON body with arguments
auth header with our Bearer token
finch_request: this one requires some explaining. Req is a high level HTTP Library built on top of the lower level Finch HTTP library. With this option, we can configure the Finch request handling manually using a function callback. That’s what we’re doing here.

The Finch.stream/5 function takes a callback function where we define how to handle streamed data, headers and the status. Each time returning the response or an error. In our case we handle status by setting the status on the response, headers by setting the headers, and data by calling our callback (cb) function with said data.

The Chat Completions API will return the streamed data in lines with format data: <JSON>\n\ndata: <JSON>... until it returns a data: [DONE] which is a little strange since Server Sent Events end when the connection closes but so it goes! We handle this in our decode_body which checks for empty strings, and [DONE] via pattern matching.

We are also appending the data to the body just in case we want to use it after the stream is complete.

And that’s basically it! We can call our function like so

    OpenAI.gpt_stream("How do I train a cat to shake hands?", fn data ->
  IO.inspect(data)
end)

  

You can do whatever you want with the data, such as sending the data to a pid or PubSub.broadcast it, but I will leave that as an exercise to the reader!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Tensors and Nx, are not just for machine learning

2023-05-10T00:00:00+00:00

This post is about using NX with Elixir and how easily it can be done for every day math! If you want to deploy your Phoenix LiveView app right now, then check out how to get started. You could be up and running in minutes.

The Elixir community has a sleeping giant brewing with Nx, and not just for Machine Learning. Nx allows you to describe nearly any numerical operation, which can then be run in an optimized environment for such operations. If you read the Nx README it describes itself as:

… a multidimensional tensors library for Elixir with multistaged compilation to the CPU/GPU.

It continues with a very impressive list of features that for sure make sense to people coming from Numerical programming, but what about us regular Elixir developers? Typically, the BEAM is not a place to do efficient math programming, but Nx changes the landscape.

You may not be looking specifically to handle large or multidimensional tensors—but consider this: if you can express it in tensor form, you can take advantage of Nx for faster calculations.

In this post, we’re going to try it out and walk through what’s possible. The first thing I recommend you do is either open up an elixir script or a Livebook with the following line:

    Mix.install([
    :nx
])

  

What is a Tensor?

A tensor is a Data Structure that is wildly flexible. It is used to wrap computations over matrixes or anything that can be described as a matrix. For example, this is a tensor:

    Nx.tensor(42)

  

    #Nx.Tensor<
s64
42
>

  

This simply means that we have a tensor of type, (64)bit (s)igned integer, with value 42. There is an implied matrix of 1x1 but that isn’t relevant here. And we can do math with this tensor!

    Nx.tensor(42)
|> Nx.add(10)

    #Nx.Tensor<
s64
52
>

  

You’ll notice that Nx was able to take our integer 10, coerce it to the correct shape, then do the addition, returning a new tensor with value 52. What happens if we try and add a float?

    Nx.tensor(42)
|> Nx.add(10.5)

    #Nx.Tensor<
f32
52.5
>

  

Here it coerced the tensor into a (32)bit (f)loat, then did the addition, returning a new tensor.

So far I hope you’re still with me, let’s take it up a notch, lists of integers!

    Nx.tensor([1, 2, 3, 42, 5])

  

    #Nx.Tensor<
s64[5]
[1, 2, 3, 42, 5]
>

  

Here we have a (64)bit (s)igned integer vector of 1x5, a vector is just a single row or column of a matrix. You can verify the matrix shape here using the Nx.shape/1 function. Let’s see what happens when we do some math on this tensor:

    Nx.tensor([1, 2, 3, 42, 5])
|> Nx.add(10)

    #Nx.Tensor<
s64[5]
[11, 12, 13, 52, 15]
>

  

Is it what we expected? Nx saw we had a tensor with shape {5} and a tensor of shape {} and automagically “broadcasted” the tensor into shape {5}. Which is equivalent to:

    Nx.add([1,2,3,42,5], [10, 10, 10, 10, 10])

  

Most operations will do their best to coerce the types and shapes of tensors into the correct shape for you. Nx has functions for just about all basic element-wise math functions.

To try and relate this back to functional programming, we’re essentially doing an Enum.map/2 over the first vector and adding the value to each item:

      Enum.map([1,2,3,42,5], fn n -&gt; n + 10 end)

  

In fact, for all of our 1 dimensional vectors we could implement everything using Enum functions.

And it doesn’t stop there, we also have aggregate functions as well:

    Nx.tensor([1, 2, 3, 42, 5])
|> Nx.mean()

    #Nx.Tensor<
f32
10.600000381469727
>

  

If we just stopped now, we’d already have a powerful tool for working with simple lists of data. And thanks to Nx this could be optimized to run on a CPU or GPU and in parallel with essentially zero work from us. Just as an example, here is a flow of computation that you could write with Nx:

    k = Nx.Random.key(1) # Random key seed

# roll 1000 dice with a shape of 1x1000
{dice_rolls, _} = Nx.Random.randint(k, 1, 6, shape: {1000})

Nx.divide(Nx.sum(dice_rolls), 1000) # 2.96
# or
Nx.mean(dice_rolls) # 2.96

Nx.multiply(dice_rolls, dice_rolls) # {1000} vector dice_roll*dice_roll
|> Nx.mean() # 10.85

  

Really, the sky is the limit here!

Multiple Dimensions

Here is where we finally catch up to the baseline description for Nx, it’s not limited to our basic 1 dimension, it can work with N dimensions, let’s start with 2:

    Nx.tensor([
    [1, 2, 3, 4, 5], 
    [12, 22, 32, 42, 52]
])

  

    #Nx.Tensor<
s64[2][5]
[
    [1, 2, 3, 4, 5],
    [12, 22, 32, 42, 52]
]
>

  

Here we have a (64)bit (s)igned integer with 2 rows and 5 columns, {2, 5} aka a 2x5 matrix. All the same functions apply here that we used above, and this is where we really start using the matrix optimizations of the GPU/CPU.

You can see how this might be useful with images, which are Width X Height sized matrixes of color values. This is what the Image library will help you with: give it an image and it will help you build a tensor that you can manipulate!

Conclusion

A tensor is a high level way of describing mathematical operations over N dimensions. Be it 0, 1, or 1000, we can operate over these tensors in a performant way with a single interface.

And that is truly amazing.

We are really only scratching the surface of what’s possible using Nx. The old adage of “Beam is not good at math” is no longer true, as the real possibilities are now endless. Just having this ability has already created an explosion of projects such as:

Bumblebee which builds a high level interface around pre-trained AI and ML models.
Scholar gives us a tool chest of classic machine learning/statistical tensor functions for doing high level math.
Explorer works with large datasets without exploding our memory usage
Image high level library for doing image manipulation, using Nx it enables classification and low level operations on raw image data.

Each of these is pretty math and science focused, but the only way to grow this list is for someone to take the first step!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Developing after mix ecto.dump

2023-05-08T00:00:00+00:00

We’re Fly.io. We run apps for our users on hardware we host around the world. This post is about how to continue development after you’ve run mix ecto.dump and deleted old migration files. Fly.io happens to be a great place to run Phoenix applications. Check out how to get started!

Problem

At some point, you or your team decided to cut the dead weight of many old migrations and instead run mix ecto.dump and generate a priv/repo/structure.sql file.

Now you want to bootstrap a new staging server, dev environment, or just get setup with a clean database. You have a chicken-and-the-egg problem. Your app can’t boot without some initial DB schema and it doesn’t have the old migrations to do that. Also, you can’t IEx into your running app to execute the SQL because it can’t run without the schema.

Looking for a solution when deploying to a clean database in a production-like environment? Check out this post!

This is the problem we need to solve: How do we continue development an app that relies on a structure.sql file generated from mix ecto.dump and we are missing older migrations?

Solution

Thankfully the problem is easy to fix, but the solution isn’t obvious.

Assuming the mix ecto.dump command was used, we have a priv/repo/structure.sql file generated for us. That file should be committed to our version control system.

Next, we’ll update our project’s mix.exs file and tweak the ecto.setup task. It probably looks something like this: (Reformatted for readability)

    defp aliases do
  [
    # ...
    "ecto.setup": [
      "ecto.create",
      "ecto.migrate",
      "run priv/repo/seeds.exs"
    ],
    "ecto.reset": ["ecto.drop", "ecto.setup"],
    # ...
  ]
end

  

The alias we care about is ecto.setup.

We’ll add a step between "ecto.create" and "ecto.migrate". We want to call mix ecto.load. However, we’ll use a couple of the flags to make it safely continue when it detects our schema is already loaded.

The command we want is ecto.load --skip-if-loaded --quiet. After updating , it should look like this:

    "ecto.setup": ["ecto.create", "ecto.load --skip-if-loaded --quiet", "ecto.migrate", "run apps/core/priv/repo/seeds.exs"],

  

Now when we develop or test against a clean DB, it creates the DB, loads the priv/repo/structure.sql file into our database, then runs any additional migrations needed before continuing on.

Awesome! We’ve got a slick solution! Problem solved.

Do you need to solve deploying to a clean database in a production-like environment? Check out this post!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Building a Drag-and-Drop List with LiveView and SortableJS

2023-05-03T00:00:00+00:00

LiveView hooks provide a powerful way to seamlessly integrate JavaScript libraries into LiveView applications. In this post, we’ll integrate SortableJS to build a list component with draggable items. Fly.io is a great place to run your Phoenix LiveView applications! Check out how to get started!

In this post, we’ll create a List component with draggable and droppable elements functionality. We’ll use some components from core_components.ex to design the List component, and then add the necessary logic to implement its behavior. The end result will be a List component that looks something like this:

For building it:

We’ll define the skeleton of a List component using some of the new components defined in core_components.ex
We’ll add Sortable JS to our LiveView app, and we’ll create a Hook to interoperate with the library from our component.
We’ll add Tailwind variants to improve the appearance of list items when they are being dragged

Let’s go for it!

Defining a live_component

We begin by defining a :live_component called ListComponent, along with its two main callbacks, render/1 and update/2:

    defmodule ComponentsExamplesWeb.ListComponent do
  use ComponentsExamplesWeb, :live_component

  def render(assigns) do
    ~H"""
    <div class="bg-gray-100 py-4 rounded-lg">
      <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">

      </div>
    </div>
    """
  end

  def update(assigns, socket) do
    {:ok, assign(socket, assigns)}
  end
end

  

Next, we add some elements to our component. As we saw above, the component has three main sections: 1) a header with the list title and an input for adding new elements, 2) the list of draggable elements, and 3) a button for clearing the list.

Let’s add the component header, which contains a text input to add new elements to the list:

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
+      <.header>
+        <%= @list_name %>
+        <.simple_form
+          for={@form}
+          phx-change="validate"
+          phx-submit="save"
+          phx-target={@myself}
+        >
+          <.input field={@form[:name]} type="text" />
+          <:actions>
+            <.button class="align-middle ml-2">
+              <.icon name="hero-plus" />
+            </.button>
+          </:actions>
+        </.simple_form>
+     </.header>    
    </div>
  </div>
  """
end

  

As you may have noticed, we used some function components that we haven’t defined. This is because these components are defined in the core_components.ex file that is generated when you create a new project with phoenix 1.7.x

Now the part we’re interested in today: the items list!

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
      <.header>
        ...
      </.header>   
+      <div id={"#{@id}-items"}>
+        <div
+          :for={item <- @list}
+          id={"#{@id}-#{item.id}"}
+          class="..."
+        >
+          <div class="flex">
+            <button type="button" class="w-10">
+              <.icon
+                name="hero-check-circle"
+                class={[
+                  "w-7 h-7",
+                  if(item.status == :completed, do: "bg-green-600", else: "bg-gray-300")
+                 ]}
+              />
+            </button>
+            <div class="flex-auto block text-sm leading-6 text-zinc-900">
+              <%= item.name %>
+            </div>
+            <button type="button" class="w-10 -mt-1 flex-none">
+             <.icon name="hero-x-mark" />
+           </button>
+         </div>
+       </div>
      </div> 
    </div>
  </div>
  """
end

  

Just below the .header, we define a container for the list items. The first child of this container includes the attribute :for={item <- @list}, to iterate over the items in the list. For each item in the list some elements are rendered: an icon, the name of the item, and a button for removing the item from the list.

Uff, a lot of new elements in a few lines, huh. Now, how do we drag and drop the list items? Let’s see it!

Adding SortableJS to our LiveView app

There is an existing JavaScript library called SortableJS that provides drag-and-drop functionality for elements inside an HTML tag. Let’s add SortableJS to our LiveView application!

Here’s how you can add it:

Go to the SortableJS source repository and locate the sortable.js file.
Copy the sortable.js file to the /assets/vendor/ directory in your Phoenix project.

Next, we need to import the Sortable library. You can do this by adding the following line at the top of the app.js file:

  import {Socket} from "phoenix"
  import {LiveSocket} from "phoenix_live_view"
  import topbar from "../vendor/topbar"
+ import Sortable from "../vendor/sortable"

Once the Sortable library is imported, we can use it in our component by defining a Hook in the same file:

    let Hooks = {}

Hooks.Sortable = {
  mounted(){
    let sorter = new Sortable(this.el, {
      animation: 150,
      delay: 100,
      dragClass: "drag-item",
      ghostClass: "drag-ghost",
      forceFallback: true,
      onEnd: e => {
        let params = {old: e.oldIndex, new: e.newIndex, ...e.item.dataset}
        this.pushEventTo(this.el, "reposition", params)
      }
    })
  }
}

let liveSocket = new LiveSocket("/live", 
                   Socket, 
                   {params: {_csrf_token: csrfToken}, hooks: Hooks}
                 )

  

After the list is rendered and the LiveView is done mounting, we need to set up the Sortable object that will handle the dragging and dropping magic. This is done inside the mounted callback, where we create a Sortable object and configure some options such as the animation speed and CSS classes to be applied to the selected item.

However, the most important part is defined inside the onEnd callback, which is executed once we drop the element being dragged. We create a constant params that contains an object with the positions of the element before and after being dragged, as well as a mysterious element ...e.item.dataset.

This last element contains all the values of custom attributes (e.g. data-x) that have been defined in the draggable HTML element. More on this in a few minutes.

Once the necessary parameters are defined, we send a reposition event to our component using pushEventTo.

Now let’s see what we need to add to our component.

Using SortableJS in LiveView components

We add a couple of lines to our component:

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
      <.header>
        ...
      </.header>   
-     <div id={"#{@id}-items"}>
+     <div id={"#{@id}-items"} phx-hook="Sortable" data-list_id={@id}>
        <div
          :for={item <- @list}
          id={"#{@id}-#{item.id}"}
          class="..."
+         data-id={item.id}
        >
          ...
        </div>
      </div> 
    </div>
  </div>
  """
end

  

To the container of the list, we add the hook that we defined earlier, along with the data-list_id attribute that helps identify the list where elements are being dropped. We also include the data-id custom attribute in each container of the list elements, which helps to identificate the element being dragged.

Our component is now ready for use! Let’s prepare the necessary assigns and render it inside a LiveView.

      defmodule ComponentsExamplesWeb.ShoppingListLive do
  use ComponentsExamplesWeb, :live_view

  def mount(_params, _session, socket) do
    list = [
      %{name: "Bread", id: 1, position: 1, status: :in_progress},
      %{name: "Butter", id: 2, position: 2, status: :in_progress},
      %{name: "Milk", id: 3, position: 3, status: :in_progress},
      %{name: "Bananas", id: 4, position: 4, status: :in_progress},
      %{name: "Eggs", id: 5, position: 5, status: :in_progress}
    ]

    {:ok, assign(socket, shopping_list: list)}
  end

  def render(assigns) do
    ~H"""
    <div id="lists" class="grid sm:grid-cols-1 md:grid-cols-3 gap-2">
      <.live_component
        id="1"
        module={ComponentsExamplesWeb.ListComponent}
        list={@shopping_list}
        list_name="Shopping list"
      />
    </div>
    """
  end
end

  

Let’s see what we’ve achieved:

Tada! Our component works!… or not?

Do you remember that the Hook sends some data to our component when an element dropped into its final position? We still need to handle that event:

    def handle_event("reposition", params, socket) do
  #Put your logic here to deal with the changes to the list order 
  #and persist the data
  IO.inspect(params)
  {:noreply, socket}
end

  

Let’s see what parameters are sent to our component:

    %{
  "id" => "1",
  "new" => 1,
  "old" => 0,
}

  

Yay! We fixed the small error we had!

Did you notice any other issues in the video above? There is one small detail that needs improvement - the formatting of the ghost CSS that is displayed in the new position where we might drop the element, as well as the appearance of the dragged HTML element.

Let’s see how we can elegantly fix this.

Formatting draggable list items

Fixing this detail is simple. We’ve already specified the CSS classes to format the drop placeholder and the dragged article. However, instead of defining new CSS classes, we can leverage the ones already defined by Tailwind. The only issue is that we can only specify a single CSS class in the Sortable object’s configuration, not a list of classes.

Thankfully, there’s a neat solution: we can use Tailwind variants to extend the CSS classes we previously specified. To do this, we only need to add a few lines to our tailwind.config.js file:

 plugin(({addVariant}) => addVariant("phx-no-feedback", [".phx-no-feedback&", ".phx-no-feedback &"])),
 plugin(({addVariant}) => addVariant("phx-click-loading", [".phx-click-loading&", ".phx-click-loading &"])),
 plugin(({addVariant}) => addVariant("phx-submit-loading", [".phx-submit-loading&", ".phx-submit-loading &"])),
 plugin(({addVariant}) => addVariant("phx-change-loading", [".phx-change-loading&", ".phx-change-loading &"])),
+plugin(({addVariant}) => addVariant("drag-item", [".drag-item&", ".drag-item &"])),
+plugin(({addVariant}) => addVariant("drag-ghost", [".drag-ghost&", ".drag-ghost &"])),

And make a small modification to our component:

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
      <.header>
        ...
      </.header>   
      <div id={"#{@id}-items"} phx-hook="Sortable" data-list_id={@id}>
        <div
          :for={item <- @list}
          id={"#{@id}-#{item.id}"}
          class="..."
          data-id={item.id}
+         class="
+         drag-item:focus-within:ring-0 drag-item:focus-within:ring-offset-0
+         drag-ghost:bg-zinc-300 drag-ghost:border-0 drag-ghost:ring-0
+         "
        >
-          <div class="flex">
+          <div class="flex drag-ghost:opacity-0">
           ...
          </div>
        </div>
      </div> 
    </div>
  </div>
  """
end

  

Let’s see how it looks:

Tada! looks better huh?

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix LiveView app close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

Bonus: multiple lists

What if we want to drag items between different lists? It is an option that Sortable already has, we just have to configure it:

    Hooks.Sortable = {
+ let group = this.el.dataset.group
  mounted(){
    let sorter = new Sortable(this.el, {
+     group: group ? group : undefined,
      animation: 150,
      delay: 100,
      dragClass: "drag-item",
      ghostClass: "drag-ghost",
      forceFallback: true,
      onEnd: e => {
        let params = {old: e.oldIndex, new: e.newIndex, ...e.item.dataset}
        this.pushEventTo(this.el, "reposition", params)
      }
    })
  }
}

  

With this option, you can move items between all lists that share the same group. Now we just need to add an assign :group to our component, and the :data-group attribute to the <div> where we set the Hook:

    def render(assigns) do
  ~H"""
  <div class="bg-gray-100 py-4 rounded-lg">
    <div class="space-y-5 mx-auto max-w-7xl px-4 space-y-4">
      <.header>
        ...
      </.header>   
-       <div id={"#{@id}-items"} phx-hook="Sortable" data-list_id={@id}>
+         <div
+           id={"#{@id}-items"}
+           phx-hook="Sortable"
+           data-list_id={@id}
+           data-group={@group}
+        >
           ...
          </div>
        </div>
      </div> 
    </div>
  </div>
  """
end

  

We can add as many lists as we need:

    def render(assigns) do
  ~H"""
  <div id="lists" class="grid sm:grid-cols-1 md:grid-cols-3 gap-2">
    <.live_component
      id="1"
      module={ComponentsExamplesWeb.ListComponent}
      list={@shopping_list}
      list_name="Shopping list 1"
+     group="grocery_list"
+   />
+   <.live_component
+       id="2"
+       module={ComponentsExamplesWeb.ListComponent}
+       list={@shopping_list}
+       list_name="Shopping list 2"
+       group="grocery_list"
+     />
  </div>
  """
end

  

We can also send the data to identify the destination list as part of the event parameters sent to the component, by using the e.to.dataset attribute:

    Hooks.Sortable = {
 let group = this.el.dataset.group
  mounted(){
      ...
      onEnd: e => {
-       let params = {old: e.oldIndex, new: e.newIndex, ...e.item.dataset}
+       let params = {old: e.oldIndex, new: e.newIndex, to: e.to.dataset, ...e.item.dataset}
        this.pushEventTo(this.el, "reposition", params)
      }
    })
  }
}

  

As a result, we obtain new parameters that can be used to identify the destination list and modify the event handling logic accordingly:

    %{
  "id" => "1",
  "new" => 2,
  "old" => 1,
  "to" => %{"group" => "grocery_list", "list_id" => "2"}
}

  

Discussion

In this post, we’ve learned how to set up SortableJS in our LiveView app and use it in our live components. But here’s the thing - we’re not actually doing anything with the data we send from the client to the server yet. We still need to figure out how to persist our elements and the position changes.

In addition, currently we’re keeping all the elements of the list in memory, which is an area for potential improvement. Fortunately, LiveView provides some incredible tools that allow us to effectively manage large collections without needing to keep them all in server memory at all times. So, the next step is to optimize our component using LiveView Streams.

Minimum Viable ChatGPT Plugin

2023-04-24T00:00:00+00:00

This is a post about creating a ChatGPT Plugin with Elixir Phoenix. If you want to deploy your Phoenix App right now, then check out how to get started. You could be up and running in minutes.

Problem

You just got access to the ChatGPT Plugin beta, and you want to hook up your database of knowledge to the AI GPT4, so it can help you and customers navigate your complex world. You’ve never done that, and all of their docs are in Python…

Solution

A ChatGPT plugin is essentially a manifest file and an OpenAPI spec, to tell ChatGPT how to consume an API. Phoenix is perfect for building the kind of APIs that ChatGPT consumes, so let’s see how to create a minimum viable plugin using Phoenix and Elixir! The only prerequisite is a searchable database; this can be anything that accepts text queries and returns data you want, from SQLite full text search to a third party API like Algolia.

We can build more complex plugins for ChatGPT to consume with, but this guide walks through only the most basic example; to set a foundation for us to build on. Let’s get started with a fresh project.

Phoenix

    mix phx.new --no-assets --no-html --no-live --no-mailer --no-ecto --no-dashboard chat_gpt_plugin

  

When building a ChatGPT Plugin, we’re actually building a standard JSON API with an OpenAPI spec configuration. So we’re generating a new Phoenix Application with basically just an Endpoint and Router. If you have an existing Phoenix Application, you should be able to copy the code directly into your code base.

Next up let’s generate a JSON API:

mix phx.gen.json Search Document documents title:string body:string --no-context

* creating lib/chat_gpt_plugin_web/controllers/document_controller.ex
* creating lib/chat_gpt_plugin_web/controllers/document_json.ex
* creating lib/chat_gpt_plugin_web/controllers/changeset_json.ex
* creating test/chat_gpt_plugin_web/controllers/document_controller_test.exs
* creating lib/chat_gpt_plugin_web/controllers/fallback_controller.ex

Add the resource to your :api scope in lib/chat_gpt_plugin_web/router.ex:

    resources "/documents", DocumentController, except: [:new, :edit]

Telling the Phoenix generator to skip generating the context with the --no-context flag. Let’s first open up the controller and make some edits: removing everything except the index function, it should look like this:

    defmodule ChatGptPluginWeb.DocumentController do
  use ChatGptPluginWeb, :controller

  alias ChatGptPlugin.Search

  action_fallback ChatGptPluginWeb.FallbackController

   def index(conn, %{"query" => query}) do
      # Here is where YOU search
    documents = Search.list_documents(query)
    render(conn, :index, documents: documents)
  end
end

  

Noting that I am purposely leaving out HOW you search for documents. When I was developing this example I used SQLite Full Text Search, but you can use Postgres or a fancy Vector Database, or Elasticsearch!

Next up, we will add a route to our Router, and it should look something like this:

    defmodule ChatGptPluginWeb.Router do
  use ChatGptPluginWeb, :router

  pipeline :api do
    plug :accepts, ["json"]
  end

  scope "/api", ChatGptPluginWeb do
    pipe_through :api

    get "/gpt-search", DocumentController, :index
  end
end

  

We’ll also want to update our view code by editing document_json.ex:

    defmodule ChatGptPluginWeb.DocumentJSON do
  @doc """
  Renders a list of documents.
  """
  def index(%{documents: documents}) do
    %{data: for(document <- documents, do: data(document))}
  end

  @doc """
  Renders a single document.
  """
  def show(%{document: document}) do
    %{data: data(document)}
  end

  defp data(document) do
    %{
      title: document.title,
      body: document.body
    }
  end
end

  

I simply removed the match on %Document{} as I don’t have that. Feel free to modify this to match your model! And in terms of Phoenix specific stuff, we are done here.

Chat GPT Specifics

ChatGPT has a couple specific needs for your local plugin to work:

CORS Enabled
It needs your server to serve the following files available:
- /.well-known/ai-plugin.json
- /openapi.yaml

To add CORS we’ll need to add the cors_plug dep to our mix.exs:

      {:cors_plug, "~> 3.0"},

  

And then add a line to our endpoint.ex, cleaning up the file to look something like this:

    defmodule ChatGptPluginWeb.Endpoint do
  use Phoenix.Endpoint, otp_app: :chat_gpt_plugin

  # CORS Config for local development
  plug CORSPlug,
    origin: ["http://localhost:4000", "https://chat.openai.com"],
    methods: ["GET", "POST"],
    headers: ["*"]

  plug Plug.Static,
    at: "/",
    from: :chat_gpt_plugin,
    gzip: false,
    only: ChatGptPluginWeb.static_paths()

  if code_reloading? do
    plug Phoenix.CodeReloader
  end

  plug Plug.RequestId
  plug Plug.Telemetry, event_prefix: [:phoenix, :endpoint]

  plug Plug.Parsers,
    parsers: [:urlencoded, :multipart, :json],
    pass: ["*/*"],
    json_decoder: Phoenix.json_library()

  plug Plug.MethodOverride
  plug Plug.Head
  plug ChatGptPluginWeb.Router
end

  

I removed the LiveView mount and Session related options as we won’t be needing them for our setup.

That’s it for CORS! Let’s update our ChatGptPluginWeb.static_paths() function in our chat_gpt_plugin_web.ex file:

      def static_paths, do: ~w(.well-known openapi.yaml favicon.ico robots.txt)

  

Adding the .well-known folder and openapi.yaml file to known static paths, also removing the images and asset stuff we won’t be needing.

And finally, create an openapi.yaml file at priv/static/openapi.yaml:

    openapi: 3.0.0
info:
  title: Example Documents Search Plugin with Elixir, Phoenix, and Sqlite3 plugin.
  description: Plugin for searching docs
  version: 1.0.0
servers:
  - url: http://localhost:4000/api/chatgpt
paths:
  /gpt-search:
    get:
      operationId: searchDocuments
      summary: Search for documents 
      description: This endpoint takes a query and searches for documentation
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - query
              properties:
                query:
                  type: string
                  description: The document description to search for.
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                type: object
                properties:
                  results:
                    type: array
                    items:
                      type: object
                      properties:
                        title:
                          type: string
                          description: The document title.
                        contents:
                          type: string
                          description: The document contents.

  

This is pretty verbose as OpenAPI’s description docs tend to be, but if you read through line by line it’s fairly self-explanatory. It’s describing our API, the get /gpt-search route we made and the query parameter we expect, and finally explains the shape of the JSON we’re returning. OpenAI will use this to tell ChatGPT how to retrieve documents, if it needs them.

Finally, our priv/static/.well-known/ai-plugin.json:

{
  "schema_version": "v1",
  "name_for_human": "Documentation Search",
  "name_for_model": "doc_search",
  "description_for_human": "Example Documents Search Plugin with Elixir, Phoenix, and Sqlite3 plugin.",
  "description_for_model": "You are an expert documentation researcher, when answering questions about my documents you reference the documentation often",
  "auth": {
    "type": "none"
  },
  "api": {
    "type": "openapi",
    "url": "http://localhost:4000/openapi.yaml",
    "is_user_authenticated": false
  },
  "logo_url": "logo.png",
  "contact_email": "jason@fly.io",
  "legal_info_url": "http://localhost:4000/terms"
}

There is more detail in the docs about these various fields, but the most important one is description_for_model as this is your prompt when searching the API. I am no expert here, but the docs lay out better examples when searching.

And that is it! If you mix phx.server the running application, open up https://chat.openai.com, click the Plugins Dropdown at the top then this little “Develop your own plugin” link

And then enter your URL http://localhost:4000/ it should find it and start working! I found this step to be a little finicky, you can enable Plugin Developer Mode in the settings of the bottom left-hand side of the page. You can also check the Browser Developer Tools for console errors, which often have better error messages than the UI. One other thing to try is to directly link to the plugin http://localhost:4000/.well-known/ai-plugin.json that might have better luck.

If all goes well, you should have your logo and description showing up in the plugin list, here is one I was playing with internally here at Fly.io! When chatting with ChatGPT it will make a decision when to reference your API based on the prompt and the expected info. If your description has keywords like Fly.io it will know to query your API for more info.

Try it yourself!

I also put together a “Single File” example showing all the steps in one simple spot.

https://github.com/jeregrine/single_file_gpt_plugin

Further Considerations

You aren’t limited to text, and in fact you will likely have a better time with structured output as in this example:

https://mobile.twitter.com/sean_moriarity/status/1648085165011288064

ChatGPT is pretty good at building your specific API inputs if well-defined in your OpenAPI.yaml. On that note, I don’t recommend maintaining an openapi.yaml file by hand. There are tons of OpenAPI libraries out there such as open_api_spex that will give you a declarative API for building out your YAML without as much boilerplate.

Really the limit is your imagination on what you can build with Plugins and on top of OpenAI’s ChatGPT. If you found this helpful please reach out and let me know what you’ve built!

Fly.io ❤️ Elixir

Fly.io is a great way to run your Phoenix Apps close to your users. It’s really easy to get started. You can be running in minutes.

Deploy a Phoenix app today! →

The Phoenix Files

Using AI to Boost Accessibility and SEO

Context matters

What is alt text?

What are captions?

How can AI help?

Bring on the AI!

JSON Output

Making the request

Anthropic’s Claude

Discussion

Final Thought

Fly.io ❤️ Elixir

Resources

My Favorite new LiveView Feature

Basic visibility

Toggle multiple elements at once

Toggle with animations

Dynamic items in a list

Reusing the click event

Discussion

Fly.io ❤️ Elixir

World Page Speed Test – planet-wide elastic scale with FLAME

Measuring Page Speed

🔥🔥 Try World Page Speed Now! 🔥🔥

Going Multi-node

Elastic scaling with FLAME

Hot vs cold starts

Solving the problem vs removing the problem

Fly.io ❤️ Elixir

How Phoenix LiveView Form Auto-Recovery works

The Problem of State

Enter A Server Restart

Takeways

Fly.io ❤️ Elixir

Easy at-home AI with Bumblebee and Fly GPUs

What exactly are we talking about?

An Elixir app to host our model

Get and deploy the app

A client Elixir application to use it

Running local code on the GPU

What’s can I do next?

Setting some Bumblebee expectations

Develop as you would normally

Fly.io ❤️ Elixir

Tips and Tricks

Note

Server: Dockerfile needs all the Nvidia dependencies

Server: Turn off Bumblebee’s console progress logging

Server: Delay starting the Nx.Serving

Server: Enable clustering on the server

Client: Make it easy to reconnect to the server

Phoenix Dev Blog – Server logs in the browser console

The Problem

The Solution

Clustering Elixir From Laptop to Cloud

What do we mean by “clustering”?

Why cluster the app on my laptop with one on the server?

1. Developing AI/ML applications

2. Developing distributed applications

3. It’s cool

How do we do it?

Conclusion

What if S3 could be a fast, globally synced, Key Value Database? That's Tigris

Enter Tigris

Experiment

Setup Tigris

Brainstorming

Durable Processess

Tables and Indexes Oh My!

Single Tenancy

Append-only tables.

Wrap-up

Fly.io ❤️ Elixir

Not every Dependency is worth it.

S3

What has aws done well?

Avoids HTTP Client Bloat

Bare bones API with few Custom Structs/API’s

What is so bad about code-gen?

What has `aws` done well?

What is so bad about `code-gen`?

LiveView’s new `handle_async/3` callback

We still use `%AsyncResult{}` even when we don’t care about the result