# rss_item

guid: https://strongly-typed-thoughts.net/blog/orphans-problemThu, 20 Sep 2018 12:30:00 GMT
title: Trying to tackle the orphans problem
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/orphans-problem
feedurl: https://phaazon.net/blog/feed
content:
<h1>Foreword</h1>
<p>The <a href="https://wiki.haskell.org/Orphan_instance#Description">orphan instance problem</a> is a well known problem that I’ve discovered while designing code in
Haskell years ago. According to the Haskell wiki:</p>
<blockquote>
<p>An orphan instance is a type class instance for class C and type T which is neither defined in
the module where C is defined nor in the module where T is defined. </p>
</blockquote>
<p>If it’s a bit overwhelming for you, here it is rephrased: this problem happens whenever you try to
create an instance of a typeclass <code>C</code> in a module that is not the module where <code>C</code> is defined for a
type <code>T</code> while you’re not in the module that actually introduces that <code>T</code> type. Since it’s not
possible to pick an instances by hand, GHC disables you from writing several instances for a same
combination of typeclass and types.</p>
<blockquote>
<p>GHC actually has a workaround for the problem: the <code>{-# OPTIONS_GHC -fno-warn-orphans #-}</code> pragma
annotation (or the <code>-fno-warn-orphans</code> compiler switch). However, this is not recommended if you
can find another way because you expose yourself to ambiguities there.</p>
</blockquote>
<p>Because Rust also has types and typeclasses (traits), it doesn’t escape the problem very much. You
still have orphan instances in Rust (called <a href="https://doc.rust-lang.org/reference/items/implementations.html#trait-implementation-coherence">Orphan Rules</a>). There’re scoped to crates though, not
modules.</p>
<p>In this blog entry, I want to explore a specific problem of orphans and how I decided to solve it in
a crate of mine. The problem is the following:</p>
<blockquote>
<p><strong>Given a crate that has a given responsibility, how can someone add an implementation of a given
trait without having to use a type wrapper or augment the crate’s scope?</strong></p>
</blockquote>
<h1>A harsh solution: type wrappers</h1>
<p>The typical workaround to this problem is to use a type wrapper. It works by encapsulating the
origin type into another layer of typing so that the compiler recognize it as a complete different
type, allowing you to implement whichever traits you want. Rust has a nice feature added to that: if
you implement the <code>Deref</code> and <code>DerefMut</code> traits, you’re given access to all the implementations of
the source type via <a href="https://doc.rust-lang.org/std/ops/trait.Deref.html">deref rules</a>, which is pretty sick. This works if you’re writing a binary,
because no one will ever use your type wrapper, only you (and your team mates, but you’ll never
<em>publish</em> the type so you’re breaking the orphan problem). However, what happens if you’re writing
a library?</p>
<p>You will expose a type that is not the source one, forcing people to wrap it again if they want to
implement another feature. If your type has a very different form from the source one, it’s okay.
However, if you want to stick to the original semantics and use cases, people might get confused,
especially whenever they will write functions that accept as arguments the source type.</p>
<h1>The problem explained in <code>splines</code></h1>
<p><a href="https://crates.io/crates/splines"><code>splines</code></a> is a crate that provides you with spline interpolation. You can interpolate values in
several dimensions with several kind of interpolators (step, linear, cosine, etc.).</p>
<p>Clearly, the scope of <a href="https://crates.io/crates/splines"><code>splines</code></a> is to provide math and curves types and functions, nothing more.
However, I use them in <a href="https://crates.io/crates/spectra"><code>spectra</code></a>, a demoscene crate of mine, in which I need them to implement
some <a href="https://crates.io/crates/serde"><code>serde</code></a> traits in order to serialize and deserialize them. Are you starting to see the
problem?</p>
<p>When I thought about where I should write the code to allow serialization / deserialization, I came
across the realization that writing it directly in <code>splines</code> would add another set of dependencies
for <em>everyone</em>, even people not using the serialization part of it. I was a bit unsatisfied with
that. So I thought about <em>“Why not just adding another crate, like <code>splines-serde</code>?”</em> Obviously,
that doesn’t work because of the orphan rules: you cannot write an implementation of a trait in a
different crate than where the type or the trait is defined. Meh.</p>
<h1>A limited solution yet a solution: feature-gated impls</h1>
<p>So I came with an idea, that is not perfect, but fits my needs pretty well. Rust has that
interesting concept of <a href="https://doc.rust-lang.org/book/first-edition/conditional-compilation.html">features</a>, allowing for conditional compilation. <code>cargo</code> supports them in
the manifest and even allows you to declare dependencies as optional. The combination of both
features and optional dependencies is key here.</p>
<h2>Step 1: the manifest</h2>
<p>Let’s have a look at the <a href="https://github.com/phaazon/splines/blob/3cd65dce54510e289e85a1a680a60f06abff5f73/Cargo.toml">current <code>Cargo.toml</code> manifest</a>
in order to get how it’s done – I’ll slice the upper part and only show the features-related part:</p>
<pre><code class="language-toml">[features]
default = [&quot;std&quot;, &quot;impl-cgmath&quot;]
serialization = [&quot;serde&quot;, &quot;serde_derive&quot;]
std = []
impl-cgmath = [&quot;cgmath&quot;]

[dependencies.cgmath]
version = &quot;0.16&quot;
optional = true

[dependencies.serde]
version = &quot;1&quot;
optional = true

[dependencies.serde_derive]
version = &quot;1&quot;
optional = true
</code></pre>
<p>Here, you can see that we are using <em>default features</em>. That means that if you depend on <code>splines</code>
in simple ways (for instance, by just giving a version string, i.e. <code>splines = &quot;0.2&quot;</code>), you’ll get
those features enabled by default. In the <code>splines</code> case, you’ll get the <code>&quot;std&quot; </code> and
<code>&quot;impl-cgmath&quot;</code> features by default.</p>
<p>Looking at the <code>&quot;std&quot;</code> feature, you can see that it doesn’t depend on anything else. That feature
will just make the whole library use the <code>std</code> crate. If you disable it, you can compile <code>splines</code>
with <code>no_std</code>.</p>
<p>The <code>&quot;impl-cgmath&quot;</code> has a dependency on <code>cgmath</code> and you can see that it depends on the <code>&quot;0.16&quot;</code>
version and that it’s optional. What it means is that if you disable default features you will not
depend on <code>cgmath</code> anymore and thus, you will not even download / compile the dependency.</p>
<p>If you look closely at that manifest, you also see a feature that must be set explicitly:
<code>&quot;serialization&quot;</code>. That feature depends on both <code>serde</code> and <code>serde_derive</code>, adding support for the
serialization code we talked earlier.</p>
<p>All of that is great and cool but how do we write the <code>impl</code>s based on that manifest?</p>
<h2>Step 2: the conditional code</h2>
<p>There’re no rules here and people will give different advices on how you should write conditional
code. I tend to remain simple as long as the project is not too complex. <code>splines</code> is a pretty
simple and small project, so let’s get things straight!</p>
<p>The first thing to do is to know how we can access the features in Rust code. This is easy: via
<em>attributes</em>. Attributes in Rust are enclosed in square brackets preceded by a dash and bang –
<code>#![]</code>. There’s an interesting attribute: <code>cfg()</code>. It gives access to the configuration of the
project, for short. It may take several kind of parameters but we’re interested in only one:
<code>feature</code>. The syntax is the following:</p>
<pre><code class="language-rust">#![cfg(feature = &quot;name_of_feature&quot;)]
</code></pre>
<p>This oneliner will evaluate the following block whenever the <code>&quot;name_of_feature&quot;</code> is set. If you want
to evaluate a block if a feature is not set, you can use the <code>not()</code> combinator:</p>
<pre><code class="language-rust">#![cfg(not(feature = &quot;name_of_feature&quot;))]
</code></pre>
<p>It’s a bit ugly, but it does the job.</p>
<p>One final and cool attribute we’ll be using: <code>cfg_attr()</code>. It takes two arguments: the first one is
a regular parameter you’d give to <code>cfg</code> that gets substituted as a boolean expression (<code>feature(…)</code>,
<code>not(feature(…))</code>, etc.) and the second one is an attribute that will get set whenever the former
argument gets substituted successfully. For instance, this:</p>
<pre><code class="language-rust">#![cfg(not(feature = &quot;std&quot;))]
#![no_std]
</code></pre>
<p>Can be rewritten more elegantly as this:</p>
<pre><code class="language-rust">#![cfg_attr(not(feature = &quot;std&quot;), no_std)]
</code></pre>
<p>For the rest, as <code>#[]</code> applies to the direct item after it, it’s easy to write conditional
<code>extern crate</code> for instance:</p>
<pre><code class="language-rust">// on no_std, we also need the alloc crate for Vec
#[cfg(not(feature = &quot;std&quot;))] extern crate alloc;

#[cfg(feature = &quot;impl-cgmath&quot;)] extern crate cgmath;

#[cfg(feature = &quot;serialization&quot;)] extern crate serde;
#[cfg(feature = &quot;serialization&quot;)] #[macro_use] extern crate serde_derive;
</code></pre>
<p>The <code>#[cfg_attr(…)</code> is even nicer when wanting to insert attributes on a type definition, as with
the <code>Key</code> type:</p>
<pre><code class="language-rust">#[derive(Copy, Clone, Debug)]
#[cfg_attr(feature = &quot;serialization&quot;, derive(Deserialize, Serialize))]
#[cfg_attr(feature = &quot;serialization&quot;, serde(rename_all = &quot;snake_case&quot;))]
pub struct Key&lt;T&gt; { … }
</code></pre>
<p>If you want <em>static ifs</em> in the actual implementation, you can cheat and use blocks as they’re
items!</p>
<pre><code class="language-rust">Interpolation::Cosine =&gt; {
  let cp1 = &amp;keys[i+1];
  let nt = normalize_time(t, cp0, cp1);
  let cos_nt = {
    #[cfg(feature = &quot;std&quot;)]
    {
      (1. - f32::cos(nt * consts::PI)) * 0.5
    }

    #[cfg(not(feature = &quot;std&quot;))]
    {
      use core::intrinsics::cosf32;
      unsafe { (1. - cosf32(nt * consts::PI)) * 0.5 }
    }
  };

  Some(Interpolate::lerp(cp0.value, cp1.value, cos_nt))
}
</code></pre>
<p>And finally, the one we were looking for to solve our orphans problem:</p>
<pre><code class="language-rust">#[cfg(feature = &quot;impl-cgmath&quot;)]
impl Interpolate for Vector2&lt;f32&gt; { … }
</code></pre>
<p>That <code>impl</code> will only exist if the <code>&quot;impl-cgmath&quot;</code> feature is set! Sweet!</p>
<p>I have another ultra cool use of attributes used along with existential <code>impl Trait</code>, but that’ll be
for another blog entry.</p>
<p>Keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/glsl_crateSun, 30 Jul 2017 00:00:00 GMT
title: Rust GLSL crate
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/glsl_crate
feedurl: https://phaazon.net/blog/feed
content:
<p>It’s been almost two months that I’ve been working on my <a href="https://crates.io/crates/glsl">glsl</a> crate. This crate
exposes a GLSL450 compiler that enables you to parse GLSL-formatted sources into memory in the form of an
AST. Currently, that AST is everything you get from the parsing process – you’re free to do whatever you want
with it. In the next days, I’ll write a GLSL writer (so that I can check that I can parse GLSL to GLSL…). I’d
love to see contributions from Vulkan people to write a SPIR-V backend, though!</p>
<p>Just for the record, the initial goal I had in mind was to parse a subset of GLSL for my
<a href="https://github.com/phaazon/spectra">spectra</a> demoscene framework. I’ve been planning to write my own GLSL-based
shading language (with modules, composability, etc. etc.) and hence I need to be able to parse GLSL sources.
Because I wanted to share my effort, I decided to create a dedicated project and here we are with a GLSL crate.</p>
<p>Currently, you can successfully parse a GLSL450-formatted source (or part of it, I expose all the intermediary
parsers as well as it’s required by my needs to create another shading language over GLSL). See for instance
<a href="https://gist.github.com/phaazon/fbe7a9c26bdea4a7262d2ea028c578ce">this shading snippet</a> parsed to an AST.</p>
<p>However, because the project is still very young, there a lot of features that are missing:</p>
<ul>
<li>I followed the <a href="https://www.khronos.org/registry/OpenGL/specs/gl/GLSLangSpec.4.50.pdf">official GLSL450 specifications</a>,
which is great, but the types are not very intuitive – some refactoring must be done here;</li>
<li>I use <a href="https://crates.io/crates/nom">nom</a> as a parser library. It’s not perfect but it does its job very well.
However, error reporting is pretty absent right now (if you have an error in your source, you’re
basically left with <code>Error(SomeVariant)</code> flag, which is completely unusable;</li>
<li>I wrote about 110 unit tests to ensure the parsing process is correct. However, there’s for now zero semantic
check. Those are lacking.</li>
<li>About the semantic checks, the crate is also missing a semantic analysis. I don’t really know what to do here,
because it’s a lot of work (like, ensure that the assigned value to a float value is a real float and not a
boolean or ensure that a function that must returns a vec3 returns a vec3 and not something else, etc.). This
is not a trivial task and because this is already done by the OpenGL drivers, I won’t provide this feature
yet. However, to me, it’d be a great value.</li>
</ul>
<p>If you’re curious, you can start using the crate with <code>glsl = &quot;0.2.1&quot;</code> in your <code>Cargo.toml</code>. You probably want
to use the <code>translation_unit</code> parser, which is the most external parser (it parses an actual shader). If you’re
looking for something similar to my need and want to parse subparts of GLSL, feel free to dig in the
<a href="https://docs.rs/glsl">documentation</a>, and especially the <code>parsers</code> module, that exports all the parsers
available to parse GLSL’s parts.</p>
<p>Either way, you have to pass a bytes slice. If your source’s type is <code>String</code> or <code>&amp;str</code>, you can use the 
<code>str::as_bytes()</code> function to feed the input of the parsers.</p>
<blockquote>
<p>Not all parsers are exported, only the most important ones. You will not find an octal parser, for instance,
while it’s defined and used in the crate internally.</p>
</blockquote>
<h1>Call to contribution</h1>
<p>If you think it’s worth it, I’m looking for people who would like to:</p>
<ul>
<li>write a GLSL to SPIR-V writer: it’s an easy task, because you just have to write a function of the
form <code>AST -&gt; Result&lt;SPIRV, _&gt;</code>, for being pragmatic;</li>
<li>test the crate and provide feedback about any error you’d find! Please do not open an issue if you have
an error in your source and find <em>“the error from the parsers is not useful”</em>, because it’s already
well established this is a problem and I’m doing my best to solve it;</li>
<li>any kind of contributions you think could be interesting for the crate.</li>
</ul>
<p>Happy coding, and keep the vibe!</p>


guid: https://strongly-typed-thoughts.net/blog/editors-in-2022Sun, 01 Jan 2023 00:00:00 GMT
title: My thoughts about editors in 2022
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/editors-in-2022
feedurl: https://phaazon.net/blog/feed
content:
<p>Today is the 1st of January, 2023. I think it’s the right moment to write another blog article about editors and 
productivity platforms. If you haven’t read my previous iterations in that series, here are some links you might want 
to have a look at (not required for this article, though):</p>
<ul>
<li><a href="https://phaazon.net/blog/editors-in-2020">My thoughts about editors in 2020</a>.</li>
<li><a href="https://phaazon.net/blog/editors-in-2021">My thoughts about editors in 2021</a>.</li>
<li><a href="https://phaazon.net/blog/neovim-plugins-stability">Neovim plugins stability</a>.</li>
<li><a href="https://phaazon.net/blog/development-environments">Development environments</a>.</li>
</ul>
<p>My take on all of that has evolved a bit over the last months / weeks. A couple of things happened and I think it’s
time for an update.</p>
<h1>My views on productivity platforms</h1>
<p>If you have read my articles about development environment, you already know what I’m talking about here. If not, 
let me do a little summary. Basically, as a software engineer, I need to use <em>tools</em> to solve problems. Those 
problems are daily little issues while working on a project:</p>
<ul>
<li>Organizing my work.</li>
<li>Searching for stuff, like a file name, some documents, discovering a code base, etc..</li>
<li>Editing code in an efficient way.</li>
<li>Building and running programs.</li>
<li>Testing programs.</li>
<li>Debugging programs.</li>
<li>Versioning my code.</li>
<li>Recording notes, journaling, creating meeting summaries, etc..</li>
<li>Manipulating program output.</li>
<li>Composing such output.</li>
<li>Viewing read-only data and grepping into it.</li>
<li>Testing an API by issuing HTTP / gRPC / whatever calls to target endpoints.</li>
<li>Using various systems such as <code>bazel</code>, <code>helm</code>, <code>kubernetes</code>, etc. and composing them with the rest.</li>
<li>Etc. etc.</li>
</ul>
<p>Whatever the tools you are using, those problems will roughly be the same. Maybe you don’t have them all, but you
will come across a couple of them on a daily basis. Let’s start talking about what most people use: IDEs.</p>
<h2>IDEs</h2>
<p>An <a href="https://en.wikipedia.org/wiki/Integrated_development_environment">IDE (Integrated Development Environment)</a>
is a software that provides a solution to a subset of the problems I mentioned above. For instance, <strong>IntelliJ 
IDEA</strong> has a solution to edit your code, go to definitions, implementations, lookup files, classes, symbols;
version your code and write Git commit messages; it has a debugger; it has a way to build and test your code; it
has a terminal so you can run random CLI commands; etc. It doesn’t solve all the problems, but clearly it solves
a subset of them. With programs like that, most of the time, you install it, and you’re good to go without 
configuring anything. <strong>JetBrains</strong> editors are famous (and loved!) for this reason. You want to work with Java?
Install <strong>IDEA</strong>. Of course, you will still find plugins to customize the experience, but the <em>vanilla</em> editor is
most of the time enough.</p>
<p>Then you have “customizable” IDEs, such as <strong>VS Code</strong>. Such softwares will often require you to download plugins
because the default experience is unlikely to have support for your language, even though it should be enough for
most people. <strong>VS Code</strong> is clearly the most famous one and it has a plugin for everything you might need (or not 
know you need!). I could have merged the two IDE sections into one, but I do make a difference in my mind because
of how <strong>JetBrains</strong> are advanced and ready-to-be-used. <strong>VS Code</strong>, whether you like it or not, is an important 
piece of software. Microsoft has made a big change with it, since most developers would agree it’s a good editor
and environment to develop in, and it helped introducing things like 
<a href="https://microsoft.github.io/language-server-protocol/">LSP</a> and 
<a href="https://microsoft.github.io/debug-adapter-protocol/">DAP</a>. You cannot trash-talk <strong>VS Code</strong> in a non-joking way,
they contributed too much and we all use their work.</p>
<h2>The rest</h2>
<p>And then, you get into the “do one thing and do it right” way of working, but it’s more complicated than that.
Historically, you would find tools such as <strong>Vim</strong>, <strong>Neovim</strong>, the <strong>git</strong> CLI, <strong>fzf</strong>, terminals, shells, TUI
applications, etc. However, as time passes, there is a trend: people tend to transform those “unit tools” into
IDEs. The terminology doesn’t really matter (whether you want to call that IDE or your own neologism). What matters
is that such tools are not minimal nor “do one thing and do it right” anymore. <strong>Neovim</strong>, for instance, is now
more a Lua interpreter in disguise of an editor, allowing to build via the plugin ecosystem, than a minimal editor.
I was told that I was wrong thinking that Lua wasn’t part of the equation since the beginning, so yeah, I was a bit
on the wrong track from the start. </p>
<p>It’s the same trend as it has been in the <strong>Vim</strong> ecosystem for so many years. Just look at plugins like 
<a href="https://github.com/tpope/vim-fugitive">vim-fugitive</a> or <a href="https://github.com/preservim/nerdtree">nerdtree</a>. In 
<strong>Neovim</strong>, you have plugins like <a href="https://github.com/williamboman/mason.nvim">mason.nvim</a> (which is
basically yet another package manager), <a href="https://github.com/folke/lazy.nvim">lazy.nvim</a> (same thing but different),
<a href="https://github.com/nvim-tree/nvim-tree.lua">nvim-tree.lua</a>, a file explorer, 
<a href="https://github.com/TimUntersberger/neogit">neogit</a>, a Git implementation in Lua, and the list goes on.</p>
<p>So, yes, I also contribute to that “plugin-based” ecosystem with <a href="https://github.com/phaazon/hop.nvim">hop.nvim</a>
and <a href="https://github.com/phaazon/mind.nvim">mind.nvim</a>, but I’ve been thinking about all that quite a lot. That adds
up to what I discuss in my article about configuration vs. scripting (basically, I think configuration should 
remain data, not code — which what scripting is about). Quoting something I said on the <strong>Neovim</strong> Matrix room, <em>“where
people see power, I see weakness”</em>. A scripting language brings lots of powerful things, such as extensibility, but
it also brings bugs, hence unstability, and a Turing-complete language, preventing the host (i.e. <strong>Neovim</strong> or even
plugins) to easily use the configuration to discover options and data, without having to standardize an API. The 
most infuriating point here to me is colorscheme. They are just scripts. Some of them are even stateful (they cache
things in <code>~/.cache/nvim</code>). So “applying a colorscheme” is not simply just switching highlights mapping: it requires
to <em>execute code</em>, which makes it super hard to reason about the colorscheme. What’s a colorscheme when you know it
can run an HTTP command?</p>
<p>I’ve been seeing a <em>lot</em> of new plugins, since I am the author and maintainer of 
<a href="https://this-week-in-neovim.org">This Week in Neovim</a>. And recently, I saw a video from 
<a href="https://github.com/tjdevries">TJ DeVries</a>: 
<a href="https://www.youtube.com/watch?v=stqUbv-5u2s">Effective Neovim: Instant IDE</a>. And that video confirms what I said
above <strong>Neovim</strong> becoming an IDE. But it also made me realize something else; TJ uses a plugin as support for 
explaining how to easily turn your favorite editor into an IDE: 
<a href="https://github.com/nvim-lua/kickstart.nvim">kickstart.nvim</a>.</p>
<p>That plugin is basically a single Lua <code>init.lua</code> script which serves as a starting init script for your configuration.
You just install it and it downloads a bunch of stuff for you, and calls all the required Lua code to set up correctly
the LSP implementation, <a href="https://tree-sitter.github.io/tree-sitter/">Tree-sitter</a>, downloading managers (yes, plural,
because you need one for plugins and one for LSPs, debuggers, linters, etc.), installing various plugins for 
completion, Git integration, surrounding pairs, etc. And then, I wondered: <em>“Newcomers are expected to run a
script that can download pretty much anything from the Internet… or write pretty much most of what it does — which is
a lot — on their own?!”</em> And the thing is that, my own Lua configuration, which is not based on 
<code>kickstart.nvim</code> since I created years ago, has been completely obsolete and I often need to go back to it to remove
or update things, especially regarding LSP, completion, snippets and all.</p>
<h1>Why I think it’s bad</h1>
<h2>Do one thing and do it right?</h2>
<p>Most people from the community I talked to disagree with my point of view regarding <strong>Neovim</strong>. For them, plugins like
<a href="https://github.com/williamboman/mason.nvim">mason.nvim</a> are amazing, because they close the gap between their 
editor and the tools required by their editor to work correctly (Mason downloads LSPs / linters / DAP 
adapters / etc.). I used Mason too, but eventually stopped using it when it started downloading a version of
<a href="https://rust-analyzer.github.io/">rust-analyzer</a> that was released <strong>years ago</strong> (that was a bug in Mason, I 
guess?). I came to the conclusion that I was depending on something doing HTTP calls to download tools that, in 
theory, could be used by other tools on my machines, and that I could also download myself very easily. In the case
of <code>rust-analyzer</code>:</p>
<pre><code class="language-sh">rustup component add rust-analyzer
</code></pre>
<p>Worse… some of those tools are actually packaged in my package manager (<code>pacman</code>), so I’m basically using a tool 
(Mason) that is doing the same thing as a package manager. As if we did not have enough package managers already.</p>
<h2>Too high and wrong expectations</h2>
<p>I then continued thinking about all those plugins (among some I use or even have created, like Mind!). Why should
I use them in <strong>Neovim</strong>? I’ve never been a file explorer guy that much, but I know about 
<a href="https://github.com/nvim-tree/nvim-tree.lua">nvim-tree.lua</a> and… and why do we have to have that in our editor? I
remember the state of mind I was in when I wrote my article about 
<a href="https://github.com/doomemacs/doomemacs">Doom Emacs</a>, which completely changed the way I think about software 
development. <strong>Emacs</strong> doesn’t belong to the “minimal editors”, nor the IDEs directly… it’s a different beast on its own,
but if I had to compare it to something else, I wouldn’t say <strong>VS Code</strong>, or <strong>Neovim</strong>, or anything else. I would
say <em>“My terminal with all of the commands I run, including an editor, a git implementation, etc.”</em> Wait a minute.
Why are we trying to push all those features as plugins into <strong>Neovim</strong>, again?</p>
<p>The reason I didn’t stick around <strong>Emacs</strong> was basically because of its runtime choice, and ultimately, its Lisp 
ecosystem. The <strong>Emacs</strong> community is one of the best I have been talking to, and they have really, really talented 
people working on insanely complex topics, like turning Elisp code into native code (via C), including Emacs itself.
But even with all those optimizations, the editor was still feeling too slow, and has a lot of background that you
can feel. All the abstraction layers, all the Lisp macros (oh no), etc.</p>
<p>Eventually, I went back to thinking about that sentence… that haunting sentence… <em>“Do one thing, and do it right.</em>”
That sentence has a lot of meaning and I think people have been tearing and bending it to align with their 
conviction, completely ignoring their biases. I read people from the Emacs community stating that yes, Emacs still
applies to that sentence, because <em>“It’s just a Lisp interpreter.”</em> But in the end, the experience users have highly
depends on the plugins, which is the same situation as with <strong>Neovim</strong>. And they have to configure their editor using
a Turing-complete language that might introduce bugs, complex statements (who loves to set up a colorscheme by
conditionally importing / requiring a Lua file located you-have-no-clue-where on your file system?)</p>
<p>Why are we trying to push all those features as plugins into <strong>Neovim</strong>, again? Why am I not trying to focus on using 
tools with a narrower scope, but ensure that the tool remains stable, powerful to use and compose well with the rest?</p>
<h1>Exploring new paths</h1>
<p>Lately, I have discovered <a href="https://helix-editor.com/">Helix</a>, a <em>“post-modern modal editor”</em>. The first thing I
noticed was that it’s different than <strong>Neovim</strong> in terms of motions. In <strong>Neovim</strong>, in normal mode, you start with a
<em>verb</em>, like deleting is <code>d</code>, and then you type a motion, like a word is <code>w</code>. So typing <code>dw</code> on your keyboard will 
delete a word. In <strong>Helix</strong>, it’s reversed. You first select with <code>w</code>, and then you apply the verb you want. 
So <code>wd</code>. At first, I thought is was a neglible difference. Then I realized how more powerful the [Helix]’ way ways.
Since you “see” the selections, you can select first and then decide what to do, or even change your mind and extend
a bit the selection. You have this nice visual feedback.</p>
<p>And then comes all the good stuff. <strong>Helix</strong> comes with those included features, <strong>requiring zero configuration</strong>:</p>
<ul>
<li>An LSP native implementation (the editor is written in Rust, so is the LSP implementation then), with all the 
features you expect, like preview popups, documentation, signatures, go-to-def, references, implementations, rename,
code actions, etc.</li>
<li>Tree-sitter support, including highlights, indents, text-objects, etc.</li>
<li>DAP support, both the feature and its UI.</li>
<li>Multi-cursors.</li>
<li>Surrounding pairs.</li>
<li>Git gutters.</li>
<li>Fuzzy finders for buffers, symbols, jump lists, diagnostics, project-wise / global grepping, etc.</li>
<li>Etc.</li>
</ul>
<p>And it has no plugin support for now (they plan on adding it at some point, but not for now). And that made me realize:
my editing experience in that editor — even though it took a couple of days to adjust the muscle memory — has been 
flawless. So yes, I miss my <code>hop.nvim</code> plugin, but I realized I could hack around by using 
<a href="https://sw.kovidgoyal.net/kitty/kittens/hints/">Kitty hints</a> until that kind of motions is built-in.</p>
<p>However, I’m just talking about editing, here. I still have that reflex of pressing <code>SPC g g</code> to bring up a Git prompt
in my editor to start committing… but <strong>Helix</strong> doesn’t have one. So I’m splitting my terminal into another window and
I use the <code>git</code> CLI. And it’s fine. Now, the way I think about it is that I could probably invest time into learning
<a href="https://github.com/jesseduffield/lazygit">lazygit</a> or anything else.</p>
<p>The point is that my editor is now minimal again. My configuration (which is public, you can find it 
<a href="https://github.com/phaazon/config/blob/master/helix/config.toml">here</a>) is mostly about 
<a href="https://bepo.fr/wiki/Accueil">bépo</a> remappings and some very minimal aesthetics changes. The configuration is data
(it’s just a TOML file), and I don’t have to worry about stability anymore since I’m not using any plugin. The fact
that I have an amazing editing experience (even better, honestly, due to the selection then verb principle, 
multi-cursors, out-of-the-box LSP and Treesitter experience, including completion) is just the perfect fit for what
I want.</p>
<h1>Yes, but</h1>
<p>But there’s a catch. See, <strong>Helix</strong> is about editing. If you like to have a file explorer in your editor, the way I
would recommend looking at <strong>Helix</strong> is that <em>you cannot have it in Helix and you should probably use a proper,
standalone file explorer, or consider another alternative like Neovim</em>. If there is something you would like to add
to <strong>Helix</strong>, you have to open a PR and write some Rust code. You cannot extend it on your own, as it doesn’t have
plugin.</p>
<p>To me, that’s great, because it means the scope is under the responsibility of the <strong>Helix</strong> Team. And I love that. I 
love it because it’s <em>easy</em> to think about the features of my editor. It’s easy because I don’t have to keep 
fearing something break because of an incompatibility between a plugin and the version of the editor I’m running
(or even two plugins between each other).</p>
<p>And honestly, contributing using a statically and strongly typed language (Rust) feels so much <em>sounder</em> to me than
using something like Lua. You can benefit from all the tests of the code base, use the API without any ABI 
conversion in between, and catch bugs at compile time instead of waiting for them to crawl up at runtime.</p>
<h1>So what does it mean for “productivity platforms?”</h1>
<p>My view on that hasn’t changed since my last articles. I still think that the terminal needs to be revamped and go
into a direction similar to Emacs. I have started a project a couple of months ago that tries to explore that. 
Basically, I’m making something that is not a terminal, a shell nor editor. It’s “a platform”, with primitives like
tree views, item lists, read-only / read-write buffers, virtual text, popups, command outputs, cells, etc. And it
comes with no way to edit text or file explorers, or anything.</p>
<p>Then, applications targetting that platform can use all those primitives to compose and now the features are
emergent. A text  editor then uses the buffer, virtual text and popup primitives, for instance. A file explorer would 
use the tree view primitive. An LSP client could be a daemon that attaches to edit buffers. Etc. etc.</p>
<p>That’s the dream tool I would love to see, and I still think that Emacs is the closest thing to that, but it comes
with too much legacy to me. And it’s unlikely that my experiment will ever be mature enough to be usable or even 
used. But you know, I like experimenting. A cool project to play with is <a href="https://www.nushell.sh/">nushell</a>. It’s far
from being that dream platform of mine, but it has some very interesting ideas for composing commands that I think
are worth mentioning.</p>
<p>And no, I don’t want <strong>Helix</strong> to become such a platform. Nor <strong>Emacs</strong> to get rid of its legacy and become it. Nor
<strong>Neovim</strong>. I want to keep playing with <strong>Helix</strong> and using it for what it is (and shines for!): editing code. If my
dream platform ever exists, whether it’s mine or someone else’s, I will probably move away from <strong>Helix</strong> to whatever
that platform provides. But such a change would require a standardization, such as <code>stdout</code> and <code>stdin</code>, but with all
those primitives I mentioned. And I’m not sure whether such a thing will or can exist.</p>
<h1>And Helix? Is it good?</h1>
<p>I’m not going to give my <em>complete</em> opinion on <strong>Helix</strong> just yet. I have been using it for a couple of days only,
and at the time writing, it still has a lot of missing parts / experimental ones. For instance, its DAP support is
experimental, so I can’t judge. What I plan to do is to stick to it and move away from “making my own IDE in an 
editor” and instead enjoying composing tools on the CLI. Then, when I have enough hindsight, I will give a fair review
of <strong>Helix</strong>.</p>
<h1>What about <code>mind.nvim</code>, <code>hop.nvim</code> and This Week in Neovim?</h1>
<p>So, about <code>mind.nvim</code>, I plan on rewriting the plugin as a standalone tool so that I can use it whatever the editor.
It will probably do things like <code>git</code> when you run <code>git commit</code> (opening <code>$EDITOR</code>), but I’m still not sure exactly
how I’m going to make it. Maybe I’ll get in touch with people from <a href="https://charm.sh/">Charm</a> and rewrite it using some 
of their work? I still haven’t thought about it, it’s too early.</p>
<p>About <code>hop.nvim</code>, I plan on continuing maintaining it and fixing bugs, even though I haven’t been very active around <code>Hop</code>
lately. The reason is mainly a lack of spare-time.</p>
<p>As for <strong>This Week in Neovim</strong>… I honestly do not know. I discussed the project with some people from the <strong>Neovim</strong> core 
team, and I’m a bit stuck. On one side, the community has received it pretty well, given the amount of upvotes I have
on each week release on Reddit; the comments; the appreciation issues on GitHub, etc. I know people have been
enjoying my work, and I’m happy they do.</p>
<p>On the other side, the core team doesn’t seem to have noticed it that much, and none of their members approached me
to talk about it. So I’m not sure what to think. The community enjoys TWiN a lot; the core team doesn’t really care.
Then I need to think about <em>exhaustion</em>: I’m really tired of maintaining TWiN. </p>
<p>See, the idea is to communicate, every week, about what has happened in the <strong>Neovim</strong> world, whether it’s core or 
plugins. What I had initially in mind was to <em>bootstrap</em> the couple of first releases and let people adopt and
contribute to it. On the 2nd of January 2023 is released <strong>TWiN #25</strong>, which means that I’m currently on a 
25 weeks streak. What it basically tells is that, every week (most of the time Sundays), I skim Reddit, GitHub 
projects, man pages, etc. to get as much information as I can, and create a really big PR containing the weekly 
update. That PR is merged and available on the very next day (Monday) for every neovimmers to enjoy reading on Monday 
morning with a nice cup of coffee, tea or whatever you like for breakfast.</p>
<p>So every week, one person (me) spends hours skimming many projects, while what I thought would happen was that
many plugin authors would contribute once every two months a very small text to explain their new plugins / change.
The difference is massive: on one side, you have a single developer doing a big amount of work… every week. On the
other side, you would have many developers doing a very small amount of work every time they release something… which
is clearly not every week (and even then?).</p>
<p>I think I have enough distance with the project to admit I failed marketing my idea. Someone once told me that I was
basically doing free advertisement for plugin authors, which is actually true. People mention they would like to 
donate to contribute and ensure that I keep doing what I do, but I don’t want money — hosting costs me 10€/month and the
domain name is 10€/year, I can sustain that on my own. What I need is contributions. It wouldn’t cost much for a plugin author 
to open a PR to <a href="https://github.com/phaazon/this-week-in-neovim-contents/pulls">twin-contents</a> and add their
update to the upcoming week. There’s a few regular contributors, writing good PRs I rarely
need to modify. But most of the weeks are contribution-free, and it saddens me even more when I see the reaction of
plugin authors on Reddit, like <em>“Oh yeah my plugin made it to TWiN!”</em> Every time I read that, I think <em>“Great, next
time maybe they will be pushing the update themselves to help me?”</em> And most of the time, they don’t.</p>
<p>So, people started to mention that I should slow down or I will burn out. And I’m honestly pretty fed up with this
<em>read-only</em> relationship: people consume / read; they rarely contribute, even when they could contribute their own
update for their own plugins! I don’t filter out <em>anything</em>, as long as it’s about <strong>Neovim</strong> and doesn’t convey any bad
speech (you know the deal). TWiN is about new plugins, updates of existing plugins, tips of the week, blog articles, 
youtube videos, etc. Anything <strong>Neovim</strong> related produced by a member of the community. And even with the exposure of 
TWiN, people still do not contribute. Even after the big refactoring to ease contribution <a href="https://www.reddit.com/r/neovim/comments/yeo89j/twin_gets_a_new_collaboration_process_to_ease/">I announced on 
Reddit</a>. So yes, it’s
a personal failure to market my idea regarding TWiN, and I’m not sure what the next steps are.</p>
<p>Nevertheless, we all learn from mistakes and it’s important to understand them. I will collect my thougths and decide
what to do next. For the time being, I wish you all a Happy New Year, lots of great things, and a tremedous amount of
happy hacking, in your favorite editor, IDE or whether you pipe <code>echo</code> commands directly at the end of files!</p>
<p>Keep the vibes!</p>
<blockquote>
<p><a href="https://www.reddit.com/r/programming/comments/1007yq5/my_thoughts_about_editors_in_2022/">Discuss here</a></p>
</blockquote>


title: On public API and private implementation
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/public-api-private-implementation
feedurl: https://phaazon.net/blog/feed
guid:
https://strongly-typed-thoughts.net/blog/public-api-private-implementationThu, 13 Jun 2019 17:05:00 GMT

content:
<p>In this blog article, I want to explore a problem I’ve been facing from time to time in <a href="https://crates.io/crates/luminance">luminance</a>.</p>
<h1>The manual dispatch problem</h1>
<p>The idea is simple: you are writing a crate and want to expose an API to people. You want them to
know which type they can use with a given operation (let’s call it <code>update</code>). However, the actual
implementation of this update function is not performed directly by your API but is deferred to a
<em>backend</em> implementation. Some people usually like to do that with several crates; in my case, I
really don’t care and let’s think in terms of types / modules instead.</p>
<p>There are several ways to do this. Let’s start with a nobrainer.</p>
<h2>The first attempt: dynamic dispatch</h2>
<pre><code class="language-rust">use std::marker::PhantomData;

// This type serves as to reify typing information at runtime and also serves as “public contract”.
// What it means is that this type gives you all the possible logical types you can use.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
enum Type {
  Int,
  UInt,
  Float,
  Bool
}

// We will use a very small abstraction to represent the update function (and the backend).
// Var&lt;B, T&gt; is just a variable of type T represented in the backend B.
#[derive(Debug)]
pub struct Var&lt;B, T&gt; where B: ?Sized {
  name: String,
  _b: PhantomData&lt;*const B&gt;,
  _t: PhantomData&lt;*const T&gt;,
}

impl&lt;B, T&gt; Var&lt;B, T&gt; {
  // Just create a variable of a given name, whatever its type and backend.
  pub fn new(name: &amp;str) -&gt; Self {
    Var {
      name: name.to_owned(),
      _b: PhantomData,
      _t: PhantomData,
    }
  }
}

impl&lt;B, T&gt; Var&lt;B, T&gt; where B: Backend, T: Interface {
  // Update the value of the variable.
  //
  // For simplicity, I don’t include a mutable reference to the backend, because that is not the
  // topic of this blog article. However, you might want to if you want to be able to, for instance,
  // change the value in a map stored in the backend, for instance. But whatever.
  pub fn update(&amp;mut self, value: &amp;T) {
    B::update(self, value);
  }
}

// The interface (i.e. public API) trait. This serves as a “anything implementing that can be used
// with the API.” It simply associates a Type that is, if you remember, an exhaustive list of types
// that can be used (and reified at runtime).
trait Interface {
  const TY: Type;
}

impl Interface for i32 {
  const TY: Type = Type::Int;
}

impl Interface for u32 {
  const TY: Type = Type::UInt;
}

impl Interface for f32 {
  const TY: Type = Type::Float;
}

impl Interface for bool {
  const TY: Type = Type::Bool;
}

// The backend (i.e. private implementations) trait. An implementation should implement this trait
// to implement the actual update of a variable for a given context / backend. Notice the Interface
// constraint on the T type parameter.
trait Backend {
  fn update&lt;T&gt;(var: &amp;mut Var&lt;Self, T&gt;, value: &amp;T) where T: Interface;
}

impl Backend for () {
  fn update&lt;T&gt;(var: &amp;mut Var&lt;Self, T&gt;, _: &amp;T) where T: Interface {
    // We can reify the type of the variable at runtime thanks to the associated constant value.
    // However, you can see that we have no way to actually inspect the value. This is due to the
    // fact our update function is universally quantified on T: we cannot observe T (i.e. know its
    // exact type) from within the implementation. That might be a problem.
    match T::TY {
      Type::Int =&gt; println!(&quot;setting {} to int&quot;, var.name),
      Type::UInt =&gt; println!(&quot;setting {} to unsigned int&quot;, var.name),
      Type::Float =&gt; println!(&quot;setting {} to float&quot;, var.name),
      Type::Bool =&gt; println!(&quot;setting {} to bool&quot;, var.name),
    }
  }
}
</code></pre>
<p>Let’s dig in. The idea of this solution is to have a trait, <code>Interface</code>, that is used to create a
set of types that can be used in the API with the <code>update</code> function on the <code>Var</code> type. The
implementation is deferred to a backend via the <code>Backend</code> trait, that contains the interface of
the implementation. Basically, the <code>Var::update</code> function will select at compile-time which
backend to use, that will in its turn <em>observe</em> the type of the variable at runtime — see the
<code>match</code> block. This is not ideal as we will have branching. We would like a better way to do it.</p>
<h2>The second attempt: static dispatch</h2>
<p>Instead of dynamically dispatching the types of the variable, we can play around with it at
compile-time. That requires some changes but in the end it’s pretty clear what we need to do:</p>
<pre><code class="language-rust">use std::marker::PhantomData;

// We don’t need the Type enum anymore so I just removed it. However, even with that solution, we
// could still want to keep it around.
#[derive(Debug)]
pub struct Var&lt;B, T&gt; where B: ?Sized, T: ?Sized {
  name: String,
  _b: PhantomData&lt;*const B&gt;,
  _t: PhantomData&lt;*const T&gt;,
}

impl&lt;B, T&gt; Var&lt;B, T&gt; {
  pub fn new(name: &amp;str) -&gt; Self {
    Var {
      name: name.to_owned(),
      _b: PhantomData,
      _t: PhantomData,
    }
  }
}

impl&lt;B, T&gt; Var&lt;B, T&gt; where B: Backend, T: Interface {
  pub fn update(&amp;mut self, value: &amp;T) {
    // Notice how now, we use the update method from the Interface and not the Backend! Important
    // change, as you will see.
    T::update(self, value)
  }
}

// The interface trait loses the associated constant and gets a method, update. This method must
// work for any backend (i.e. types that implement Backend), since it’s our public facing trait.
trait Interface {
  fn update&lt;B&gt;(var: &amp;mut Var&lt;B, Self&gt;, value: &amp;Self) where B: Backend;
}

// We can then implement it for all our types by selecting manually the update_* method we want.
impl Interface for i32 {
  fn update&lt;B&gt;(var: &amp;mut Var&lt;B, Self&gt;, value: &amp;Self) where B: Backend {
    B::update_i32(var, value);
  }
}

impl Interface for u32 {
  fn update&lt;B&gt;(var: &amp;mut Var&lt;B, Self&gt;, value: &amp;Self) where B: Backend {
    B::update_u32(var, value);
  }
}

impl Interface for f32 {
  fn update&lt;B&gt;(var: &amp;mut Var&lt;B, Self&gt;, value: &amp;Self) where B: Backend {
    B::update_f32(var, value);
  }
}

impl Interface for bool {
  fn update&lt;B&gt;(var: &amp;mut Var&lt;B, Self&gt;, value: &amp;Self) where B: Backend {
    B::update_bool(var, value);
  }
}

// The trait backend is now a list of methods that will be available to the Interface’s
// implementors.
trait Backend {
  fn update_i32(var: &amp;mut Var&lt;Self, i32&gt;, value: &amp;i32);
  fn update_u32(var: &amp;mut Var&lt;Self, u32&gt;, value: &amp;u32);
  fn update_f32(var: &amp;mut Var&lt;Self, f32&gt;, value: &amp;f32);
  fn update_bool(var: &amp;mut Var&lt;Self, bool&gt;, value: &amp;bool);
}

/// Implementing the backend trait now gives us a power we didn’t have back then in solution 1: we
// now can observe the type of the variable and, then, we can actually do useful things with it, as
// displaying it!
impl Backend for () {
  fn update_i32(var: &amp;mut Var&lt;Self, i32&gt;, value: &amp;i32) {
    println!(&quot;setting {} to int {}&quot;, var.name, value);
  }

  fn update_u32(var: &amp;mut Var&lt;Self, u32&gt;, value: &amp;u32) {
    println!(&quot;setting {} to unsigned int {}&quot;, var.name, value);
  }

  fn update_f32(var: &amp;mut Var&lt;Self, f32&gt;, value: &amp;f32) {
    println!(&quot;setting {} to float {}&quot;, var.name, value);
  }

  fn update_bool(var: &amp;mut Var&lt;Self, bool&gt;, value: &amp;bool) {
    println!(&quot;setting {} to bool {}&quot;, var.name, value);
  }
}
</code></pre>
<p>We change the definition of the <code>Backend</code> trait to have all the functions dispatched statically.
Then, using the <code>Interface</code> trait, we now have one information we didn’t have in the first example:
the actual, concrete type of the variable. We can then call the right function from the <code>Backend</code>
trait.</p>
<p>To sum up, because all of this is starting to be a bit confusing:</p>
<ul>
<li>The <code>Interface</code> trait is the trait used to restrict the public API (i.e. which types can be
used publicly).</li>
<li>The <code>Backend</code> trait is the trait to implement when providing the actual implementation.</li>
</ul>
<p>However, if we add more types, that solution won’t scale easily. The problem is that we have the
typing information in several places (at the <code>impl</code> level and in a static list in a trait). It would
be much easier if we could, somehow, force an <code>impl</code> to exist in another trait. Basically, I want to
remove those <code>update_*</code> and use <code>impl</code>s instead.</p>
<h2>The third and final solution: inferred static dispatch</h2>
<p>I have no idea how to call that way of doing but I like to think of it as inferred constraints.
The idea is almost the same as the second solution but instead of declaring the list of functions
that can be used in the implementation of the <code>Interface</code> trait, we will just create a generic
dependency between the <code>Interface</code> trait and <code>Backend</code>. The advantage will also be that we don’t
have to worry about the name of the function anymore, since it will be polymorphic.</p>
<p>Let’s go.</p>
<pre><code class="language-rust">use std::marker::PhantomData;

#[derive(Debug)]
pub struct Var&lt;B, T&gt; where B: ?Sized, T: ?Sized {
  name: String,
  _b: PhantomData&lt;*const B&gt;,
  _t: PhantomData&lt;*const T&gt;,
}

impl&lt;B, T&gt; Var&lt;B, T&gt; {
  pub fn new(name: &amp;str) -&gt; Self {
    Var {
      name: name.to_owned(),
      _b: PhantomData,
      _t: PhantomData,
    }
  }
}

// Wouhouh, some changes here. We go back to the first solution by using the update method of the
// Backend trait, but notice how we bind the Backend trait to the Interface by using Backend&lt;T&gt;.
impl&lt;B, T&gt; Var&lt;B, T&gt; where T: Interface, B: Backend&lt;T&gt; {
  pub fn update(&amp;mut self, value: &amp;T) {
    B::update(self, value)
  }
}

// This trait is now a bit dumb and even dangerous as it’s very easy to implement it. In a perfect
// Rust world, we would have sealed traits — i.e. we could only implement it from the inside of the
// current crate.
trait Interface {}

impl Interface for i32 {}

impl Interface for u32 {}

impl Interface for f32 {}

impl Interface for bool {}

// The backend trait now has an associated type parameter that represents the variable type. What
// it means is that we will be able to implement the update function for all types of our choices…
// and still observe the type, since from inside the update function, it’s not universally
// quantified anymore!
trait Backend&lt;T&gt; {
  fn update(var: &amp;mut Var&lt;Self, T&gt;, value: &amp;T);
}

impl Backend&lt;i32&gt; for () {
  fn update(var: &amp;mut Var&lt;Self, i32&gt;, value: &amp;i32) {
    println!(&quot;setting {} to int {}&quot;, var.name, value);
  }
}

impl Backend&lt;u32&gt; for () {
  fn update(var: &amp;mut Var&lt;Self, u32&gt;, value: &amp;u32) {
    println!(&quot;setting {} to unsigned int {}&quot;, var.name, value);
  }
}

impl Backend&lt;f32&gt; for () {
  fn update(var: &amp;mut Var&lt;Self, f32&gt;, value: &amp;f32) {
    println!(&quot;setting {} to float {}&quot;, var.name, value);
  }
}

impl Backend&lt;bool&gt; for () {
  fn update(var: &amp;mut Var&lt;Self, bool&gt;, value: &amp;bool) {
    println!(&quot;setting {} to bool {}&quot;, var.name, value);
  }
}
</code></pre>
<p>This solution is quite interesting because of the use of the type parameter in the <code>Backend</code> trait.
It enables you to implement the <code>Backend</code> trait and still observe (i.e. <em>know</em>) the type of the
variable you’re playing with. Compare with the first solution, where we were completely generic on
the <code>T</code> type.</p>
<p>In <a href="https://crates.io/crates/luminance">luminance</a>, I use the last solution to allow a clear distinction between a set of public types
and a set of matching implementations. Notice in the last solution the <code>Interface</code> trait, which is
now reduced to something pretty dumb. It would be easy for anyone to implement it for their own
types and then implement <code>Backend&lt;TheirType&gt; for TheirBackend</code>. Rust doesn’t offer a way to have
<em>sealed traits</em> so far, so my current solution to this is to mark the trait <code>unsafe</code> (to scare
people and tell them not to implement the trait). However, a clear and first-class citizen language
construct for this would be highly appreciated.</p>
<p>That’s all for me today. If you have any question about such a design, I’d be happy to answer your
questions on Reddit, as usual.</p>
<p>Keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/kak-tree-sitterFri, 09 Jun 2023 19:20:00 GMT
title: Kakoune design analysis with kak-tree-sitter
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/kak-tree-sitter
feedurl: https://phaazon.net/blog/feed
content:
<p>I want to talk about Kakoune in this blog article, and more specifically, its UNIX design. See, in my 
<a href="https://github.com/helix-editor/helix/blob/master/LICENSE">previous</a> blog post, I explained why I love
<a href="https://helix-editor.com/">Helix</a>, and why I love <a href="https://kakoune.org/">Kakoune</a>. The design philosophy of <strong>Kakoune</strong>
really is excellent, but I want to put more relief now. Especially, I will write this blog article along the line of
<a href="https://github.com/phaazon/kak-tree-sitter">kak-tree-sitter</a>, which is a UNIX server and daemon I’ve been writing to
add support of <a href="https://tree-sitter.github.io/tree-sitter/">tree-sitter</a>.</p>
<h1>Kakoune and the shell</h1>
<p>Because Kakoune doesn’t have a plugin interface, you are limited to what is called <em>kakspeak</em>, which is basically what
you can type in the command line (the <code>:</code> line). Contrary to other editors like Vim, everything you type in that <code>:</code>
line can be laid out in a file (a <code>.kak</code>) and be interpreted directly. Another cool aspect about that is that you can
just select some part of a <code>.kak</code> file, and simply type <code>:^r.&lt;cr&gt;</code> – in Kakoune, the <code>.</code> is the selection register. That
will execute the Kakoune command.</p>
<p>This property is really good, but still, you are limited in what you can do. Among the command you will be running:</p>
<ul>
<li><code>declare-user-mode</code>, to declare your own mode (yes, like insert mode, normal mode, etc.). Yes it’s a default / core
feature and yes it’s excellent.</li>
<li><code>set-face</code> to declare / update highlighting groups.</li>
<li><code>execute-keys</code> and <code>evaluate-commands</code> to execute keys as if the user typed them, in sandboxed / isolated 
environments. This is incredibly powerful, as you can run some commands that creates, updates registers, move the 
cursor around, etc. but since it’s sandboxed, as soon as the command is done, you get back to the state you were in
before issuing the commands.</li>
<li>And more.</li>
</ul>
<p>There is nothing to write plugins… except one thing.</p>
<h2>String expansions</h2>
<p>Kakoune has this concept of <em>expanding strings</em>. It’s the same feature as in any other editor: some strings can contain
special keywords and identifiers to replace their content with what they hold. For instance, in your shell, it’s very
likely that this string will contain your username: <code>&quot;$USER&quot;</code>, or this will be the current year <code>&quot;$(date +%Y)&quot;</code>.
However, this will remain verbatim: <code>'$(date +%Y)'</code>. The reason is that, in a shell, <code>&quot;</code> expands while <code>'</code> doesn’t.</p>
<p>Kakoune has the same mechanism, but the syntax is different, and Kakoune has different <em>source</em> of expansions. For
instance, <code>%opt{foo}</code> will be replaced why the <code>foo</code> option, that can be set with <code>set-option</code>. <code>%val{timestamp}</code>
contains the current timestamp of the buffer, etc. etc.</p>
<p>There is one interesting expansion that Kakoune has: <code>%sh{}</code>. This is <em>shell expansion</em>. It will execute its content
inside a thin shell. For instance, try open Kakoune and enter in the command line, something like <code>:echo %sh{date +%Y}</code>.
You can see where we are going here. We can use that to run arbitrary shell commands.</p>
<p>Kakoune is monothreaded, so when you run a command in a shell, it blocks until the shell command finishes. On its own,
it’s not that bad. It forces us to call short-living shell commands.</p>
<p>This mechanism allows us to run external programs, but it doesn’t tell us how we can call back Kakoune.</p>
<h1>Kakoune and UNIX sockets</h1>
<p>Kakoune is monothreaded, but it’s concurrent. It listens on a UNIX socket Kakoune commands that any external programs
can send, via the <code>kak -p</code> interface.</p>
<blockquote>
<p>I initially tried to send content to the UNIX socket programmatically, but it wasn’t planned for that, and hence hit
issues while doing so. It hurts to say that but you have to spawn a programm running <code>kak -p</code>; forget about writing
directly into the UNIX socket for now.</p>
</blockquote>
<p>So, with this mechanism, we can:</p>
<ol>
<li>Run a short-living program via <code>%sh{}</code> expansion to talk to, for instance, a server, quickly accepting our request
and treating asynchronously.</li>
<li>Then, once the request is handled, send back the response to Kakoune by running a <code>kak -p</code> program, using the UNIX
socket.</li>
</ol>
<p>The cool thing is that Kakoune follows a server/client architecture and has a <em>session identifier</em> (you must pass it
to <code>kak -p</code>). You can retrieve that value with <code>%val{session}</code> — and inside <code>%sh{}</code> expansion, it’s available as an
environment variable <code>$kak_session</code>.</p>
<p>And here you have it. The formula I’ve been using successfully to add <strong>tree-sitter</strong> support to Kakoune. Whenever we
need to highlight the buffer again, simply craft a small request to send to the <code>kak-tree-sitter</code> local server and
immediately get control back (so that the UI doesn’t freeze). Then, at some point, the highlight request is computed and
arrives via <code>kak -p</code> in Kakoune.</p>
<p>But you might have a question… how do I deal with the buffer content? Indeed, the only thing we can do with with <code>%sh{}</code>
is starting a program in a shell. We could do something like laying the buffer content on the shell invocation but 
that’s seriously limiting (especially on giant buffers). We could put the content of the buffer in an environment 
variable, but it would suffer from the same problem (and damn it’s so dirty). What else?</p>
<h1>The final part of the recipe: FIFOs</h1>
<p>UNIX systems have this incredible thing called <em>FIFOs</em>. I FIFO — also named <em>pipe</em> — is a special kind of file. It lives
on your filesystem (so it’s located at a given path), a bit like UNIX socket. However:</p>
<ul>
<li>Its content <em>never</em> exists on the filesystem. It’s all in-memory.</li>
<li>It’s a streaming primitive, so you cannot just write into it and assume it’s stored somewhere.</li>
<li>Think of it as a buffer between two entities: a <em>reader</em> and a <em>writer</em>.</li>
<li>When a reader reads from it, it blocks until data is available to read.</li>
<li>When a writer writes to it, it blocks until a reader is available to read.</li>
</ul>
<p>So it implements a rendez-vous buffer between a single reader and a single writer. And yes, the <code>|</code> in your shell is
using something that behind the scene. The thing is: you can create your own, and you don’t even need to write any code.
Enter the <code>mkfifo</code> program. For instance, you can try it out on your own by creating a FIFO (let’s call it <code>rdv</code>),
reading its content with <code>cat</code> first (you’ll see the <code>cat</code> process freeze, so you will need another terminal session!)
and then write content to it, for instance with <code>echo</code>:</p>
<pre><code class="language-sh">mkfifo /tmp/rdv

# in shell 1
cat /tmp/rdv

# in shell 2
echo &quot;Hello, world!&quot; &gt; /tmp/rdv
</code></pre>
<p>As seen as the <code>echo</code> starts writing, you can see <code>cat</code> return the result.</p>
<blockquote>
<p>Now replace <code>cat</code> with <code>tail -f</code> 😏.</p>
</blockquote>
<p>Anyway, Kakoune has a mechanism where it scans <code>%sh{}</code> blocks and it sees <code>$kak_command_fifo</code> and/or
<code>$kak_response_fifo</code>, it will create those FIFOs for us (and manage their lifetimes). Because we are in the shell, we
can write Kakoune commands to execute in <code>$kak_command_fifo</code>, which will be executed as soon as you’re done writing to
the FIFO, and you can read <code>$kak_response_fifo</code> from, for instance, an external program, to get more data from Kakoune.</p>
<p>This is the exact mechanism that is used to stream buffer content between Kakoune and <code>kak-tree-sitter</code>. Here’s the
Kakoune commands used to highlight a buffer:</p>
<pre><code class="language-sh"># Send a single request to highlight the current buffer.
define-command kak-tree-sitter-highlight-buffer -docstring 'Highlight the current buffer' %{
  nop %sh{
    echo &quot;evaluate-commands -no-hooks -verbatim write $kak_response_fifo&quot; &gt; $kak_command_fifo
    kak-tree-sitter -s $kak_session -c $kak_client -r &quot;{\&quot;type\&quot;:\&quot;highlight\&quot;,\&quot;buffer\&quot;:\&quot;$kak_bufname\&quot;,\&quot;lang\&quot;:\&quot;$kak_opt_filetype\&quot;,\&quot;timestamp\&quot;:$kak_timestamp,\&quot;payload\&quot;:\&quot;$kak_response_fifo\&quot;}&quot;
  }
}
</code></pre>
<p>I won’t explain the protocol into two much details, but the important part is that I start a shell to run
<code>kak-tree-sitter … -r</code>, format the request (it’s just plain JSON), and ask <code>kak-tree-sitter -r</code> to read from 
<code>$kak_response_fifo</code>. You can see in the previous line that we write to it (the <code>write $kak_response_fifo</code> basically
means to write the content of the current buffer to <code>$kak_response_fifo</code>, which is a file — everything is a file in
UNIX!).</p>
<p><code>kak-tree-sitter -r</code> is made in a way so that it exits quickly. It will read from the FIFO file and forward the request
to the <code>kak-tree-sitter</code> server, which, in turn, will move the request to a different thread so that it can quickly
mark the request done. The whole thing is then synchronous, but extremely fast. The only bottleneck we have here (and
mind it, it’s important) is <strong>that we must read from the FIFO synchronously in the shell <code>kak-tree-sitter -r</code></strong>.</p>
<h1>How’s it going?</h1>
<p>I’ve been running with this setup for weeks now, since I started <code>kak-tree-sitter</code> around the end of April, 2023. The
code you read above runs in a couple of Kakoune hooks (the same <code>kak-lsp</code> is using or very similar), which waits for a
<em>idle</em> time after the buffer is edited (in practice, 50ms after the last edit operation). And it’s already fast enough.</p>
<p>However, there is a nice optimization that we could do here. See, the only reason to start a shell is to format the 
request to send to <code>kak-tree-sitter</code>, the server. We can completely by-pass it. Instead, we could:</p>
<ul>
<li>Create the FIFO in <code>kak-tree-sitter</code>, when a new Kakoune session is created.</li>
<li>Write the requests to the FIFO handled by <code>kak-tree-sitter</code>. No shell is needed to write to files!</li>
<li>The rest is the same, since the server sends back highlighting Kakoune commands via <code>kak -p</code>.</li>
</ul>
<p>This on its own should greatly enhance performance, which is already pretty good, even with the shell. I plan on working
on this enhancement in the upcoming days.</p>
<h1>Alright, but what’s wrong then?</h1>
<p>Well, there is one thing. See, in order to support <strong>tree-sitter</strong>, I had to write this <code>kak-tree-sitter</code> server, and
write buffer contents to FIFO files. <code>kak-lsp</code> is doing the same. All of that is synchronous, and even if it’s fast,
it’s still two sequential locks of the buffer, copy of the buffer, etc. On my side, I’m currently exploring the
<code>%val{history}</code> expanded variable, which contains the textedit operations (so that I don’t have to stream a full 
buffer and diff in <code>kak-tree-sitter</code> anymore, but instead just small chunks of updates).</p>
<p>However, the problem still holds. The design of Kakoune is nice when you’re working alone with your small integration,
but when you have two big ones (LSP and tree-sitter are not small environments), I think Kakoune shows its limit.</p>
<p>We – <a href="https://github.com/krobelus">@krobelus</a> and I – discussed that matter. A middleware program or a change in how
Kakoune interfaces with external programs, especially for buffer streaming, would be greatly appreciated. But it’s not
currently the case. And even if we decide to come up with a middleware – that could act as a buffer cache / map, for
instance – we would still be writing buffer contents to different FIFOs.</p>
<p>Another thing that I’ve been thinking about a lot is… is this the right approach? Something like <strong>tree-sitter</strong> <em>seems</em>
to be ideal as an embedded library, directly inside the code of your editor. Especially since Kakoune is all about
selections, being able to have semantic selections by default seems like something pretty good. <strong>tree-sitter</strong> requires
some runtime resources (relocatable objects like <code>.so</code> / <code>.dylib</code>, queries, etc.)… and it’s the same thing with the
regular Kakoune highlighters (by default, Kakoune doesn’t have any; they come bundled up with your distribution).</p>
<p>I’m not entirely sure the approach scales very well. Of course, I still think the design is excellent, and it prevents
too much maintenance on Kakoune, which is also a good thing. For instance, this runtime resource problem is something
I’m solving in <code>kak-tree-sitter</code> (and easing with a controller tool called <code>ktsctl</code>, which can download, compile and
install grammars and queries for you), but still. The current version of <code>kak-tree-sitter</code> only supports semantic
highlighting, not text-objects just yet (but it shouldn’t be too hard to add). The state of the project is not completly
ready for people to jump in (the wiki is not written), but if you tag along on Matrix, I can help you get started.</p>
<p>Please consider <code>kak-tree-sitter</code> as an experimental project, because I’m not sure this is the right approach, even
though it’s probably the only one for Kakoune for now (I don’t think <a href="https://github.com/mawww">@mawww</a> plans on
integrating it into the core of Kakoune).</p>
<p>Keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/asynchronous_warmy_prequelTue, 08 May 2018 19:30:00 GMT
title: Asynchronous warmy: a prequel
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/asynchronous_warmy_prequel
feedurl: https://phaazon.net/blog/feed
content:
<p>Last weeks were interesting for <a href="https://crates.io/crates/warmy">warmy</a>, a crate I’ve been writing for several weeks / months
now that enables you to hot load and reload scarce resources – e.g. textures, meshes, configuration,
JSON parameters, dependency nodes, whatever. <a href="https://crates.io/crates/warmy">warmy</a> received several interesting features, among:</p>
<ul>
<li><em>Context passing</em>: it is now possible to pass a mutable reference (<code>&amp;mut</code>) to a typed context
when loading and reloading a resource. This enables per-value loadings, which is neat if you need
to add extra data when loading your resources (e.g. increment a counter).</li>
<li><em>Methods support</em>: before version 0.7, you had to implement a trait, <code>Load</code>, in order to tell
<a href="https://crates.io/crates/warmy">warmy</a> how to load a given object of a given type. This is convenient but carries a bad drawback:
if you want to represent your object with both JSON and XML for instance, you need to <em>type wrap</em>
your type so that you can <code>impl Load</code> twice. This was annoying and the type system also handed you
back an object which type was the wrapper type, not the wrapped type. This annoyance was
removed in 0.7 as you now have an extra type variable to <code>Load</code> – it defaults to <code>()</code> though –
that you can use to <code>impl Load</code> several times – think of that tag type variable as a type
representing the <em>encoding</em> or the <em>method</em> to use to load your value.</li>
<li><em>A VFS (Virtual FileSystem)</em>: the VFS makes it possible to write resource keys without caring
about their real location – e.g. <code>/splines/color_grading.json</code>. Before that, you still had to
provide a real filesystem path, which was both confusing and annoying (since you already give one
when you create a <code>Store</code>, the object that holds your resource).</li>
<li><a href="https://github.com/phaazon/warmy/blob/master/CHANGELOG.md#07">Changelog here</a>.</li>
</ul>
<p><a href="https://www.reddit.com/r/rust/comments/8fy3q4/warmy070_vfs_context_passing_reload_methods_and/">I posted on reddit</a>
in order to make people know of the new version, and interesting talks started to occur on both IRC
and GitHub. What people seem to want the most now is <em>asynchronous loading and reloading</em>. I’ve been
wanting that feature for a while too so I decided it could be a good idea to start working on it.
However, after a day of toying around, I came to the realization that I should write a small blog
post about it because I think it’s not trivial and it could help me shape my ideas.</p>
<blockquote>
<p>Note: this post is about brainstorming and setting up the frame to why and how async warmy. You
will find incomplete code, ideas and questions there. If you want to contribute to the discussion,
you’re more than welcome!</p>
</blockquote>
<h1>Synchronous versus asynchronous</h1>
<p>What does it mean to have a <em>synchronous</em> computation? What does it mean to have an <em>asynchronous</em>
one? You might find it funny, but a lot of people are still confused with the exact definition, so
I’ll try to give you more hindsight.</p>
<p>We talk about a <em>synchronous</em> task when we have to wait for it to finish before moving on to another
task. We have to wait until its completion to get the control flow back and call other functions. We
often talk about <em>blocking computations</em>, because you cannot do anything else while that computation
is running – at least on the thread this computation is running on.</p>
<p>We talk about an <em>asynchronous</em> task when you can get the control flow back without having to wait
for the task to finish. However, <strong>that doesn’t necessarily mean that the task is being executed in
parallel or concurrently</strong>. At some extent, you could easily label <a href="https://en.wikipedia.org/wiki/Generator_(computer_programming)">generators</a> as <em>asynchronous
primitives</em>, and this is what actually happens in <code>async / await</code> code: you give back the control
flow to the caller and the callee execution gets re-scheduled later. Hence, this is asynchronous
programming, yet the scheduling execution could be implemented on a single thread – hence no
parallel nor concurrency actually happen. Another example is when you perform a HTTP request: you
can send the request and instead of blocking until the response arrive, you can give the control
back, do something else, and then, at some time, handle the response. You don’t need parallelism
to do this: you need asynchronous programming.</p>
<blockquote>
<p>Note: a generalization of a generator is a <a href="https://en.wikipedia.org/wiki/Coroutine">coroutine</a>, which hands the control flow back to
another <a href="https://en.wikipedia.org/wiki/Coroutine">coroutine</a> instead of the caller when wanted.</p>
</blockquote>
<h1>What about warmy?</h1>
<p>At the time of writing this blog entry, <a href="https://crates.io/crates/warmy">warmy</a> is completely synchronous. Consider the following
example:</p>
<pre><code>extern crate serde_json;
extern crate warmy;

use std::error;
use std::fmt;
use std::fs::File;
use std::io;
use std::thread;
use std::time::Duration;
use warmy::{FSKey, Load, Loaded, Res, Store, StoreOpt, Storage};

struct JSON;

#[derive(Clone, Copy, Debug, PartialEq)]
struct Foo {
  array: [f32; 4]
}

#[derive(Debug)]
enum FooError {
  JsonError(serde_json::Error),
  IOError(io::Error)
}

impl fmt::Display for FooError {
  fn fmt(&amp;self, f: &amp;mut fmt::Formatter) -&gt; Result&lt;(), fmt::Error&gt; {
    match *self {
      FooError::JsonError(ref e) =&gt; e.fmt(f),
      FooError::IOError(ref e) =&gt; e.fmt(f),
    }
  }
}

impl error::Error for FooError {
  fn description(&amp;self) -&gt; &amp;str {
    match *self {
      FooError::JsonError(ref e) =&gt; e.description(),
      FooError::IOError(ref e) =&gt; e.description(),
    }
  }

  fn cause(&amp;self) -&gt; Option&lt;&amp;error::Error&gt; {
    match *self {
      FooError::JsonError(ref e) =&gt; Some(e),
      FooError::IOError(ref e) =&gt; Some(e),
    }
  }
}

impl&lt;C&gt; Load&lt;C, JSON&gt; for Foo {
  type Key = FSKey;

  type Error = FooError;
  
  fn load(
    key: Self::Key,
    _: &amp;mut Storage&lt;C&gt;,
    _: &amp;mut C,
  ) -&gt; Result&lt;Loaded&lt;Self&gt;, Self::Error&gt; {
    let file = File::open(key.as_path()).map_err(FooError::IOError)?;
    let array = serde_json::from_reader(file).map_err(FooError::JsonError)?;
    let foo = Foo { array };

    Ok(foo.into())
  }
}

fn main() {
  // read scarce resources in /tmp
  let ctx = &amp;mut ();
  let opt = StoreOpt::default().set_root(&quot;/tmp&quot;);
  let mut store: Store&lt;()&gt; = Store::new(opt).expect(&quot;store creation failed&quot;);

  // get our resources via JSON decoding
  let foo: Res&lt;Foo&gt; = store.get_by(&amp;FSKey::new(&quot;/foo.json&quot;), ctx, JSON).expect(&quot;foo should be there&quot;);
  let foo2: Res&lt;Foo&gt; = store.get_by(&amp;FSKey::new(&quot;/foo2.json&quot;), ctx, JSON).expect(&quot;foo should be there&quot;);

  println!(&quot;{:#?}&quot;, foo);
  println!(&quot;{:#?}&quot;, foo2);

  loop {
    // sync the resources with the disk
    store.sync(ctx);
    thread::sleep(Duration::from_millis(100));
  }
}
</code></pre>
<blockquote>
<p>Note: you can test that code by inserting <code>warmy = &quot;0.7&quot;</code> and <code>serde_json = &quot;1&quot;</code> in a <code>Cargo.toml</code>
and by creating the JSON files containing 4D arrays of numbers, like <code>[0, 1, 2, 3]</code> in
<code>/tmp/foo{,2}.json</code>.</p>
</blockquote>
<p>If you look closely at that example, you can spot two important locations when we get resources: when
we create the <code>foo</code> and <code>foo2</code> bindings (search for <code>store.get_by</code> invocations). Here, it’s important
to understand what’s really happening:</p>
<ol>
<li>First, <code>/foo.json</code> is loading on the current thread.</li>
<li>Then, when it’s loaded and cached, <code>/foo2.json</code> starts loading.</li>
</ol>
<p>We can already see a pity here: both the files are completely disconnected from each other, yet, the
second file must wait for the former to finish before starting loading. We’re not using all the
cores / threads of our CPU. So we could perfectly imagine this new scenario:</p>
<ol>
<li>First, ask to load <code>/foo.json</code> and immediately get control flow back.</li>
<li>Then, ask to load <code>/foo2.json</code> and immediately get control flow back.</li>
<li>Wait for both the resources to be available, then continue.</li>
</ol>
<p>You could even do whatever you want between (2.) and (3.). The point here is that we can run tasks
without having to wait for them to finish before starting others. The explicit <em>waiting</em> part could
be a blocking call, such as:</p>
<pre><code>let foo_task = store.get_by(…);
let foo2_task = store.get_by(…);

let foo = foo_task.wait();
let foo2 = foo2_task.wait();
</code></pre>
<blockquote>
<p>Note: this is not the foreseen or planned interface. It’s just there to illustrate what we want to
achieve there.</p>
</blockquote>
<p>The goal of this blog entry is to explore possible implementations.</p>
<h1>futures</h1>
<p>The <a href="https://crates.io/crates/futures">futures</a> crate is a very promising and interesting crate. What it provides is basically a mean
to express <em>data that might get available in the future</em>. For instance:</p>
<pre><code>fn ping(host: &amp;str) -&gt; ???
</code></pre>
<p>Here, <code>ping</code> is a function that will perform an <a href="https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol">ICMP</a> request to a given host, identified by the
function argument. Because that function might take a while (at least several milliseconds, which is
<strong>a lot</strong>), we have to make a decision:</p>
<ol>
<li>Either we decide to block the current thread until a response gets available (the host responds
to our ping and we get the packet back).</li>
<li>Or either we decide to free the control flow from the <code>ping</code> function and do something else
until the response arrive.</li>
</ol>
<p>If you’ve followed all what I said since the beginning of this blog post, you might have noticed that
(1.) is synchronous and (2.) is asynchronous (also notice that we haven’t talked about parallelism
yet!).</p>
<p>With (1.), we would write <code>ping</code>’s signature like this:</p>
<pre><code>fn ping(host: &amp;str) -&gt; Result&lt;PingResponse, NetworkError&gt;
</code></pre>
<p>With (2.), we would write it like this, using the <code>futures</code> crate:</p>
<pre><code>fn ping(host: &amp;str) -&gt; impl Future&lt;Item = PingResponse, Error = NetworkError&gt;
</code></pre>
<blockquote>
<p>Note: the syntax <code>impl Trait</code> is called <em>conservative impl trait</em> and its description can be found
in the <a href="https://github.com/rust-lang/rfcs/blob/master/text/1522-conservative-impl-trait.md">corresponding RFC on impl Trait</a>.</p>
</blockquote>
<p>So what you basically do here is to send a non-blocking request and get its result in a non-blocking
way. Sweet! How does that apply to <a href="https://crates.io/crates/warmy">warmy</a>?</p>
<h1>futures applied to warmy</h1>
<p>Asynchronous <a href="https://crates.io/crates/warmy">warmy</a> should have the following properties:</p>
<ul>
<li>When you ask for a resource to load, you immediately get the control flow back.</li>
<li>Hot-reloading mustn’t block either, as you will do it in the <code>Store::sync</code> function.</li>
<li>Because of the asynchronous nature of those computations, the <em>context passing</em> now gets a bit
tricky: we cannot handle a mutable reference on our context anymore, because otherwise we would
block the main thread. We need to share the context in a smart way.</li>
<li>Finally, all parallel computations should be ran on some kind of <em>green thread</em> so that we don’t
have to worry about destroying the OS threads amount.</li>
</ul>
<h2>Asynchronous loading</h2>
<p>The <a href="https://docs.rs/warmy/0.7.2/warmy/load/struct.Storage.html#method.get">current definition of the <code>Store::get</code> function</a>
is:</p>
<pre><code>pub fn get&lt;K, T&gt;(
  &amp;mut self, 
  key: &amp;K, 
  ctx: &amp;mut C
) -&gt; Result&lt;Res&lt;T&gt;, StoreErrorOr&lt;T, C&gt;&gt;
where
  T: Load&lt;C&gt;,
  K: Clone + Into&lt;T::Key&gt;
</code></pre>
<p>We want something like this:</p>
<pre><code>pub fn async_get&lt;K, T&gt;(
  &amp;mut self, 
  key: &amp;K, 
  ctx: &amp;mut C
) -&gt; Result&lt;impl Future&lt;Item = AsyncRes&lt;T&gt;, Error = StoreErrorOr&lt;T, C&gt;&gt;, StoreErrorOr&lt;T, C&gt;&gt;
where
  T: Load&lt;C&gt;,
  K: Clone + Into&lt;T::Key&gt;
</code></pre>
<p>Woosh. It gets hard to read – eventually, <a href="https://github.com/rust-lang/rfcs/blob/master/text/1733-trait-alias.md">trait aliases</a> will help us there. We can see the use of
the <code>futures</code> crate here, in the return type:</p>
<pre><code>impl Future&lt;Item = AsyncRes&lt;T&gt;, Error = StoreErrorOr&lt;T, C&gt;&gt;
</code></pre>
<p>Which reads as <em>“An asynchronous resource of type <code>T</code> available in the future or an error due to the
loading of <code>T</code> and <code>C</code> or due to the store”</em>.</p>
<p>However, something important to notice is that we miss a crucial point here: we actually <em>want a
parallel execution</em>. We could start try by defining a small state machine to step through the
process of loading a resource. That could be:</p>
<pre><code>enum LoadState&lt;K, T, C&gt; {
  Pending(K, Shared&lt;C&gt;), // resource loading pending
  Loading(Receiver&lt;AsyncRes&lt;T&gt;&gt;), // resource currently loading
  Done, // resource loaded
}
</code></pre>
<p>This small state machine can be stepped through by the following process:</p>
<pre><code>fn start_load&lt;K, T, C&gt;(key: K, ctx: &amp;mut C) -&gt; impl Future&lt;Item = LoadState&lt;K, T, C&gt;&gt; {
  LoadState::Pending(key, ctx.share_somehow())
}

impl&lt;K, T, C&gt; Future for LoadState&lt;K, T, C&gt; {
  type Item = AsyncRes&lt;T&gt;;

  type Error = StoreErrorOr&lt;T, C&gt;;

  fn poll(&amp;mut self, ctx: &amp;mut Context) -&gt; Result&lt;Async&lt;Self::Item&gt;, Self::Error&gt; {
    match *self {
      LoadState::Pending(..) =&gt; {
        // start the loading
        let (sx, rx) = oneshot::channel();
        let task = Task::new(move || {
          // load the resource somehow
          let res = …;
          sx.send(res);
        });

        // spawn a new task to load the resource and update the state machine
        ctx.spawn(task);
        *self = LoadState::Loading(rx);
        Ok(Async::NotReady)
      }

      LoadState::Loading(ref mut rx) =&gt; {
        match rx.map(|res| {
          *self = LoadState::Done;
          res
        }).poll()?
      }

      LoadState::Done =&gt; panic!(&quot;poll called after future has arrived&quot;)
    }
  }
}
</code></pre>
<p>That code might be incomplete or not typecheck completely (not tested), but the idea is there. I
still don’t know whether this is the good way to use <code>futures</code> or whether I should run a single, long
computation without the small state machine. I just don’t know yet.</p>
<h2>Asynchronous reloading</h2>
<p>This part is tightly bound to how the <code>Store::sync</code> will behave. Now that I tink about it, maybe it’d
be more ergonomic / elegant / simpler to have an event loop living on a separate thread that would
benefit from <code>epoll</code> and all those fancy asynchronous primitives to deal with that. That would result
in no more by-hand synchronization, since it’d be done by the event loop / reactor / whatever you
want to name it.</p>
<p>Currently, I have not settled up on any decision regarding reloading.</p>
<h2>Asynchronous context passing</h2>
<p>This will be definitely a tricky part as well, because what was before:</p>
<pre><code>store.sync(&amp;mut ctx);
</code></pre>
<p>will then become something like:</p>
<pre><code>store.sync(ctx.share());
</code></pre>
<p>Also, something that might happen is that because the synchronization will be done in an asynchronous
way, you will need to ensure that you do not try to synchronize the same resource several times.</p>
<p>Finally, if you take into account the previous section, the user might not even synchronize by hand
anymore, so the context will have to be shared anyway and moved to the event loop. Argh. That doesn’t
seem simple at all! :)</p>
<h2>Parallel execution</h2>
<p>One final aspect of asynchronous <a href="https://crates.io/crates/warmy">warmy</a> is obviously how we’re going to <code>poll</code> the various <code>Future</code>s
generated by the <code>get</code> functions. One first, naive idea would be to run those in OS threads. However,
imagine that you’ll be loading plenty of resources (on a big and release project, it’s easy to
imagine several thousands of resources loading in parallel). You don’t want to allocate that many OS
threads but instead rely on a green threading solution… which I haven’t found yet. People on IRC
advised to have a look at <a href="https://crates.io/crates/futures-cpupool">futures-cpupool</a>, but I’d like to have a complete solution set before
deciding anything.</p>
<p>I will post this blog post on reddit with hope that people will come up with ideas and I’m sure
enlighting ideas on how to cope with all of this asynchronous properties I want to push to <a href="https://crates.io/crates/warmy">warmy</a>.</p>
<p>Thanks for having read me, and keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/reverse_prompt_git_stashSun, 16 Aug 2015 00:00:00 GMT
title: Never forget your git stashes again!
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/reverse_prompt_git_stash
feedurl: https://phaazon.net/blog/feed
content:
<p>It’s been a while I’m experiencing issues with <code>git stash</code>. If you don’t know that command yet,
<code>git stash</code> is used to move all the changes living in your <em>staging area</em> into a special place: the
<em>stash</em>.</p>
<p>The <em>stash</em> is a temporary area working like a stack. You can push changes onto it via <code>git stash</code>
or <code>git stash save</code>; you can pop changes from top with <code>git stash pop</code>. You can also apply a very
specific part of the stack with <code>git stash apply &lt;stash id&gt;</code>. Finally you can get the list of all
the stashes with <code>git stash list</code>.</p>
<p>We often use the <code>git stash</code> command to stash changes in order to make the working directory clear
again so that we can apply a patch, pull some changes, change branch, and so on. For those purposes,
the <em>stash</em> is pretty great.</p>
<p>However, I often forget about my stashes – I know I’m not the only one. Sometimes, I stash something
and go to cook something or just go out, and when I’m back again, I might have forgotten about what
I had stashed, especially if it was a very small change.</p>
<p>My current prompt for my shell, <a href="http://www.zsh.org/">zsh</a>, is in two parts. I set the <code>PS1</code>
environnment variable to set the regular prompt, and the <code>RPROMPT</code> environnment variable to set a
reversed prompt, starting from the right of the terminal. My reversed prompt just performs a <code>git</code>
command to check whether we’re actually in a <code>git</code> project, and get the current branch. Simple, but
nice.</p>
<p>I came up to the realization that I could use the exact same idea to know whether I have stashed
changes so that I never forget them! Here’s a screenshot to explain that:</p>
<p><img src="http://phaazon.net/pub/git_stash_shell.png" alt="" /></p>
<p>As you can see, my prompt now shows me how many stashed changes there are around!</p>
<h1>The code</h1>
<p>I share the code I wrote with you. Feel free to use it, modify it and share it as well!</p>
<pre><code># …

function gitPrompt() {
  # git current branch
  currentBranch=`git rev-parse --abbrev-ref HEAD 2&gt; /dev/null`
  if (($? == 0))
  then
    echo -n &quot;%F{green}$currentBranch%f&quot;
  fi

  # git stash
  stashNb=`git stash list 2&gt; /dev/null | wc -l`
  if [ &quot;$stashNb&quot; != &quot;0&quot; ]
  then
    echo -n &quot; %F{blue}($stashNb)%f&quot;
  fi

  echo ''
}

PS1=&quot;%F{red}%n%F{cyan}@%F{magenta}%M %F{cyan}%~ %F{yellow}%% %f&quot;
RPROMPT='$(gitPrompt)'

# …
</code></pre>
<p>Have fun!</p>


guid: https://strongly-typed-thoughts.net/blog/smoothie_bezierTue, 30 Jun 2015 00:00:00 GMT
title: smoothie-0.3, Bézier curves and new user interface
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/smoothie_bezier
feedurl: https://phaazon.net/blog/feed
content:
<h1>Bezier curves</h1>
<p>It’s been a while I’ve been wanting to do that. Now it’s done!
<a href="https://hackage.haskell.org/package/smoothie">smoothie</a>, a Haskell package for
dealing with curves and splines, updated to version
<a href="https://hackage.haskell.org/package/smoothie-0.3">0.3</a>.</p>
<p>That version introduces several changes. If you’re a good programmer, you might
already have noticed that the major version got incremented. That means there’re
compatibility breaking changes. If you don’t know what I’m talking about, you
should definitely read
<a href="https://wiki.haskell.org/Package_versioning_policy">this</a>.</p>
<p>The first – non-breaking – change is that the package now supports
<a href="https://en.wikipedia.org/wiki/B%C3%A9zier_curve">Bézier</a> interpolation! I’ve
been reading about <strong>Bézier curves</strong> for a while because there’re very present
and important for animation purposes – think of <strong>Blender</strong>. Feel free to
dig in the documentation on hackage for further details.</p>
<p>The second – breaking – change is that the interface has changed, especially the
implementation of <em>splines</em>. However, the interface is now simpler and doesn’t
require a lot of change in your code if you’ve been using older versions.</p>
<p>Feel free to read the
<a href="https://hackage.haskell.org/package/smoothie-0.3/changelog">CHANGELOG</a> for
technical hints.</p>
<p>As always, tell me what you think of the library, and keep the vibe!</p>


guid: https://strongly-typed-thoughts.net/blog/hop-2022Mon, 18 Jul 2022 14:26:00 GMT
title: Useful Hop in 2022
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/hop-2022
feedurl: https://phaazon.net/blog/feed
content:
<p>Lately, I have received a PR on <a href="https://github.com/phaazon/hop.nvim">hop.nvim</a> where the committer wanted to implement
a feature… that was already there. I realized that even with a great documentation, a wiki, a changelog, official
announcements, people still don’t read the documentation / feature list and implement things. The problem is that I have
to reject their contributions and they lose time (and I do as well, because I have to review contributors’ code).</p>
<p>So here’s a quick article I can link in the super fresh <a href="https://this-week-in-neovim.org/">TWiN</a>, website I released a
couple of days ago and that I hope will help with that kind of issues.</p>
<h1>What can you do with Hop</h1>
<p>Hop has been around for some time now. I created it back in February 2021, so roughly one year and a half. It has
changed and evolved quite a lot ever since. New hint algorithm, new commands, a <em>very big</em> number of new options and
features. I really recommand people to read the embedded documentation of Hop. Yes, completely. It took me time to write
in a way that it will take you a couple of minutes only to read. So yes, do read it. Get the habit to read the embedded
documentation of the plugins you install, especially after new updates. Or read <a href="https://this-week-in-neovim.org/">TWiN</a> from now on 😁.</p>
<h2>The wiki, the wiki, the wiki!</h2>
<p>Yes, Hop has a wiki. It’s available <a href="https://github.com/phaazon/hop.nvim/wiki">here</a> and you should read a couple of
pages from it:</p>
<ul>
<li><a href="https://github.com/phaazon/hop.nvim/wiki/Commands">The “default” commands</a>.</li>
<li><a href="https://github.com/phaazon/hop.nvim/wiki/Advanced-Hop">Mastering advanced Hop</a>. This part contains a couple of
examples showing you how to use the various Hop Lua functions and options to create a really advanced motion platform.</li>
<li><a href="https://github.com/phaazon/hop.nvim/wiki/Configuration">All the configuration options</a>.</li>
</ul>
<p>Because I know some people will not click on those links and will still open PRs trying to implement features that are
already there, here is a non-exhaustive list of advanced things you can already do, out of the box, with Hop.</p>
<h2>Recreate <code>f</code>, <code>F</code>, <code>t</code> and <code>T</code></h2>
<p>It’s in the wiki and super simple. For that, you just need three options:</p>
<ul>
<li><code>direction</code>, which must be set with <code>require'hop'.HintDirection.BEFORE_CURSOR</code> or
<code>require'hop'.HintDirection.AFTER_CURSOR</code>.</li>
<li><code>current_line_only</code>, a boolean.</li>
<li><code>hint_offset</code>, a super power allowing you to offset the actual jump off the jump target by a given number of
characters.</li>
</ul>
<p>So <code>f</code> is basically:</p>
<pre><code class="language-lua">require'hop'.hint_char1({
  direction = require'hop.hint'.HintDirection.AFTER_CURSOR,
  current_line_only = true
})
</code></pre>
<p><code>F</code> is then:</p>
<pre><code class="language-lua">require'hop'.hint_char1({
  direction = require'hop.hint'.HintDirection.BEFORE_CURSOR,
  current_line_only = true
})
</code></pre>
<p>And in the same idea, <code>t</code> is:</p>
<pre><code class="language-lua">require'hop'.hint_char1({
  direction = require'hop.hint'.HintDirection.AFTER_CURSOR,
  current_line_only = true,
  hint_offset = -1
})
</code></pre>
<p>And <code>T</code>:</p>
<pre><code class="language-lua">require'hop'.hint_char1({
  direction = require'hop.hint'.HintDirection.BEFORE_CURSOR,
  current_line_only = true,
  hint_offset = -1
})
</code></pre>
<h2>Jump at the end of things</h2>
<p>Hop already can already understand the concept of <em>beginning</em> and <em>end</em> of a jump target. It does not make sense with
all of them, but for most, it can. That is implemented with the <code>hint_position</code> option, which has to be one of:</p>
<ul>
<li><code>require'hop.hint'.HintPosition.BEGIN</code></li>
<li><code>require'hop.hint'.HintPosition.MIDDLE</code></li>
<li><code>require'hop.hint'.HintPosition.END</code></li>
</ul>
<p>So if you want to jump to the <em>end</em> of words:</p>
<pre><code class="language-lua">require'hop'.words({
  hint_position = require'hop.hint'.HintPosition.END,
})
</code></pre>
<p>You can even jump <em>right after</em> the end of a word!</p>
<pre><code class="language-lua">require'hop'.words({
  hint_position = require'hop.hint'.HintPosition.END,
  hint_offset = 1
})
</code></pre>
<h2>Callbacks!</h2>
<p>You can even use Hop to do something with the jump target that doesn’t imply jumping, like running a Lua function with
the actual column and line where the hint is. It’s done with <code>require'hop'.hint_with_callback</code>.</p>
<p>Here, no example but an exercise for you: try to use <code>require'hop'.hint_with_callback</code> to send a notification (via
<code>vim.notify</code>) displaying the jump target chosen by the user by hinting words. Tips: you will need to find the jump
target generator for words.</p>
<h2>Final recommendation</h2>
<p>Please, feel free to read <code>:h hop</code>. It contains everything I just explained and so much more, like Hop extensions, multi
windows, etc. Have fun hopping around!</p>


guid: https://strongly-typed-thoughts.net/blog/osx_alMon, 29 Jun 2015 00:00:00 GMT
title: Mac OS X support in al-0.1.4
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/osx_al
feedurl: https://phaazon.net/blog/feed
content:
<h1>Support for Mac users!</h1>
<p>This will be a short announcement about
<a href="https://hackage.haskell.org/package/al">al</a>, a <strong>Haskell</strong> wrapper to the
<strong>OpenAL</strong> C library.</p>
<p>Currently, the wrapper has been successfully tested on Linux – at least it works
well on my <em>Archlinux</em> distro. I made a little program that reads from an <em>.ogg</em>
file and streams the PCM signal to the wrapper – see
<a href="https://hackage.haskell.org/package/libvorbis">libvorbis</a> for further details.
I’ll release the program later on if I find the spare time.</p>
<p>The wrapper might also work on <em>Windows</em> as long as you have
<a href="http://www.freedesktop.org/wiki/Software/pkg-config/">pkg-config</a> installed.
I’d be very happy with feedback from people working on <em>Windows</em>. I don’t want
anyone be put apart with my packages.</p>
<p>However, I’ve never tested the wrapper on <em>Mac OS X</em>. I guessed it wouldn’t work
out of the box because <em>Mac OS X</em> doesn’t use regular libraries to compile and
link – that would have been too easy otherwise, and hell, <em>think different
right?</em> They use something called a <a href="https://developer.apple.com/library/mac/documentation/MacOSX/Conceptual/BPFrameworks/Concepts/WhatAreFrameworks.html">framework</a>.
It’s possible to include a framework in a <strong>Haskell</strong> project by fulfilling
the <code>frameworks</code> field in the <em>.cabal</em> file. I received a simple patch to do
that – <a href="https://github.com/mtolly/al/commit/fef897083c89899b72c2e026a7831f9c48710b5c">here</a>, and I merged it upstream.</p>
<p>Then, <em>Mac OS X</em> is now officially supported. The release version is the
<a href="https://hackage.haskell.org/package/al-0.1.4">0.1.4</a>.</p>
<h1>About stackage</h1>
<p>There’s something else I’d like to discuss. Quickly after the first release of
<a href="https://hackage.haskell.org/package/al-0.1.0.0">al</a>, I decided to push it onto
<a href="https://www.stackage.org">stackage</a>. Unfortunately, there’s a problem with the
package and <em>Ubuntu</em>. For a very dark reason, <em>Ubuntu</em> doesn’t expose
anything when invoking <code>pkg-confg --cflags</code>, even if the files are there – on
<em>Ubuntu</em> they can be found in <code>/usr/include/AL</code>.</p>
<p>That’s very silly because I don’t want to hardcode the location – it might be
something else on other Linux distro. The problem might be related to the
<strong>OpenAL</strong> support in <em>Ubuntu</em> – the <em>.pc</em> file used by <code>pkg-config</code> might be
incomplete. So if you experience that kind of issue, you can fix it by passing
the path to your <strong>OpenAL</strong> headers:</p>
<pre><code>cabal install al --extra-include-dirs=/usr/include/AL
</code></pre>
<p>If <strong>OpenAL</strong> is installed somewhere else, consider using:</p>
<pre><code>find / -name al.h
</code></pre>
<p>I’ll do my best to quickly fix that issue.</p>


title: Pre luminance-1.0.0 and random thoughts
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/pre-luminance-n-random-thoughts
feedurl: https://phaazon.net/blog/feed
guid:
https://strongly-typed-thoughts.net/blog/pre-luminance-n-random-thoughtsFri, 26 Apr 2019 22:30:00 GMT

content:
<h1>Random thoughts on making luminance-1.0.0</h1>
<p>Hello people. It’s been weeks I have started to work on luminance-1.0.0. For a brief recap,
<a href="https://crates.io/crates/luminance">luminance</a> is a graphics crate that I originally created in Haskell, when I ripped it off from a
<em>demoscene engine</em> called <a href="https://github.com/phaazon/quaazar">quaazar</a> in order to make and maintain tiner packages. The Rust port was
my first Rust project and it became quickly the default language I would develop graphics
applications in.</p>
<blockquote>
<p>So if you’re interested, yes, <a href="http://hackage.haskell.org/package/luminance">luminance in Haskell</a>
still exists but I maintain it for strictly minimal support (i.e. support compiler bumps, for
instance). The default and official language to use, if you’re interested by <a href="https://crates.io/crates/luminance">luminance</a>, is Rust.</p>
</blockquote>
<p>Another important thing I have to explain is the current feature list of <a href="https://crates.io/crates/luminance">luminance</a> and what it’s
going to be soon. Currently, <a href="https://crates.io/crates/luminance">luminance</a>’s goals are (according to the
<a href="https://docs.rs/luminance/0.30.1/luminance/">documentation</a>):</p>
<ul>
<li>Making the <strong>unsafe</strong> and <strong>stateful</strong> OpenGL API <strong>safe</strong> and <strong>stateless</strong>.
<ul>
<li><strong>Unsafe</strong> means that OpenGL performs operations by relying on the fact you know what you’re
doing: pointer arithmetic, memory alignment, etc. Lots of situations in which you can
literally <em>RIP in pepperonis</em>.</li>
<li><strong>Stateful</strong> means that lots of OpenGL functions expect a context to be in a special state for
a correct behavior. For instance, when uploading texels to the GPU’s VRAM, the texture that
must receive the texels must be bound to a texture unit. If you forget to do
that, you get a weird behavior that is seen at runtime and not always correctly interpreted.
<strong>It is sometimes hard to know why and what something has gone wrong.</strong> It makes also a <em>pain
in the @!#=$</em> as soon as you want to share work on several threads.</li>
</ul>
</li>
<li>Providing a simple and easy interface; that is, exposing core concepts without anything
extra – just the bare stuff.
<ul>
<li>Most low-level graphics library are hard for newcomers to grasp knowledge about. <a href="https://crates.io/crates/gfx-hal">gfx-hal</a>
is an excellent graphics crate (so far, I think it’s the one everyone is recommending
everyone in the Rust gamedev community) but it is hard to play with it when you don’t know
either OpenGL 4.5 or Vulkan — and all the required concepts might be a bit overwhelming at
first.</li>
<li>Even though <a href="https://crates.io/crates/luminance">luminance</a> is still a pretty low-level API (you have renderbuffers,
framebuffers, shader stages, etc.), the interface <strong>is opinionated</strong> and — in my opinion —
easier to work with. Because it is my idea and my design, the experience might be slightly
different from e.g. <a href="https://crates.io/crates/gfx-hal">gfx-hal</a>. It’s neither bad or good, just different. That also responds
to the folks who don’t understand why I’m still maintaining <a href="https://crates.io/crates/luminance">luminance</a>: because I want to
do graphics code my way, and more choices is always a good sign to me. Yes, we could talk
about community efforts, but OH LOOK A BIRD!</li>
</ul>
</li>
<li>Easy to read with a good documentation and set of tutorials, so that newcomers don’t have to
learn a lot of new concepts to get their feet wet.</li>
<li>Need-driven: every piece of code added in the project must come from a real use case. If you
feel something is missing, feel free to open an issue or even contribute!
<ul>
<li>This is an important aspect to me. I don’t write code for the sake of writing code (even
libraries). I have a problem and write code to solve it. So, on the surface, <a href="https://crates.io/crates/luminance">luminance</a>
might seem a bit less complete that, again, e.g. <a href="https://crates.io/crates/gfx-hal">gfx-hal</a>. And that would be likely right.
However, if someone likes the API and wants something in, I accept both issues and PRs. There
are several examples of people asking for features and having me implement them. There’s also
some folks who actually wrote the code and it’s now in <a href="https://crates.io/crates/luminance">luminance</a>!</li>
<li>Head over to <a href="https://github.com/phaazon/luminance-rs/issues">https://github.com/phaazon/luminance-rs/issues</a> if you want to start discussing
the feature set.</li>
</ul>
</li>
</ul>
<p>That’s the current situaton. A very important other aspect is that <a href="https://crates.io/crates/luminance">luminance</a> was designed, in the
first place, to be <em>backend agnostic</em>. I quickly decided not to go this way and stick to a simpler
design with OpenGL only.</p>
<blockquote>
<p>A comparison between <a href="https://crates.io/crates/luminance">luminance</a> and other famous crates is
<a href="https://phaazon.net/blog/luminance_comparing">available here</a>.</p>
</blockquote>
<p>However, I have — again! — changed my mind. Today, I want to have a system of backend in
<a href="https://crates.io/crates/luminance">luminance</a>. Maybe not something as complex and complete as the <strong>gfx-rs</strong> project, but something
that will allow those backends:</p>
<ul>
<li>Any <strong>OpenGL</strong> version: missing features from older versions will be implemented (if not too
much work is needed) by <a href="https://crates.io/crates/luminance">luminance</a> software patches.</li>
<li>At least one <strong>WebGL</strong> implementation and it must be WASM compatible.</li>
<li><strong>Vulkan</strong>.</li>
</ul>
<p>I really do not target <strong>Metal</strong>, <strong>DirectX12</strong> nor any other graphics backend you might have in
mind. However, if you think it’s possible to do so, then we should discuss about it. Nevertheless,
as I said above, <a href="https://crates.io/crates/luminance">luminance</a> is need-driven: as I don’t need <strong>DirectX12</strong>, I will not work on it
unless someone REALLY needs it and provides a PR (or at least enough for me to work on it… including
beers).</p>
<h1>What to expect</h1>
<p><strong>luminance-1.0.0</strong> will ship… as soon as possible. The current feature set is huge, though:</p>
<ul>
<li>Add the <code>&quot;std&quot;</code> feature-gate. This will help use <a href="https://crates.io/crates/luminance">luminance</a> with specific contexts, especially
for demoscene or size-limited devices. Currently, the work on <code>no_std</code> has been saved for later
as <a href="https://github.com/rust-lang/rust/issues/51540">I had hit a bug last time I checked</a>.</li>
<li>Change the <code>Tess</code> interface. Now, a <code>TessBuilder</code> must be used. This allows for three major
enhancements:
<ul>
<li>Vertex instancing.</li>
<li>Deinterleaved memory.</li>
<li>Builder pattern.</li>
</ul>
</li>
<li>Enhance the quality and variety of examples.</li>
<li>Introduce the <strong>luminance-derive</strong> crate. This crate provides several proc-macros used to
implement various <code>unsafe</code> traits easily, without writing any <code>unsafe</code> code. Among those:
<ul>
<li>The <code>Vertex</code> trait, used to create vertex types.</li>
<li>The <code>Semantics</code> trait, used to create vertex semantics types.</li>
<li>The <code>UniformInterface</code> trait, used to create shader <em>uniform interfaces</em>.</li>
<li>All the previous <code>macro_rules</code> are removed.</li>
</ul>
</li>
<li>Support a new dynamic way to get uniforms. Sometimes, you don’t know in advance the uniforms
your shader is going to use. When such a case arise, you might be tempted to have a minimal
<em>uniform interface</em> — if none. <em>Dynamic uniform lookups</em> allow you to query uniforms on the
fly. That has obviously a runtime cost but it can be handy for lots of situations (GUI editors,
scripting, etc.).</li>
<li>Update framebuffer code so that <em>color slots</em> and <em>depth slots</em> are easier to use.</li>
<li>Introduce the concept of <em>drivers</em>. Drivers allow code to be parametered by a type which
represents a given implementation to use for the graphics code. You can have a <code>GL33</code> driver
for <strong>OpenGL 3.3</strong>, a <code>GL40</code>, <code>GL44</code>, <code>GL45</code> etc., but you can also have <code>WebGL</code> and <code>VK10</code>.</li>
<li>Add vertex instancing. Vertex instancing allows to instantiate objects by providing instance
data directly in a <code>Tess</code> — instead of using the current method with GPU <code>Buffer&lt;_&gt;</code>. This was
asked by several people and I just couldn’t ignore such an interesting and useful feature!</li>
<li>Support for deinterleaved memory. Deinterleaved memory allows for more granularity on how data
is fetched GPU-side. With the legacy <a href="https://crates.io/crates/luminance">luminance</a> situation, data is completely interleaved,
which means that a vertex’s attributes all follow each other in a GPU buffer. This is both nice
and bad: if you need to access all attributes of a vertex, this is pretty good, since
cache-locality will be in your advantage. However, if you’re only interested in positions, for
instance, you will waste your cache lines with attributes you will never read from!
Deinterleaved memory stores each attributes in a different GPU memory region, allowing for a way
better memory scheme for such uses. People using the <a href="https://en.wikipedia.org/wiki/Entity_component_system">ECS</a> pattern might be used to
deinterleaved memory.</li>
<li>Add vertex primitive restart. This allows to use tessellation indexing modes such as
<code>TriangleFan</code> or <code>LineStrip</code> and “cut” a primitive if an index is equal to a given value. This
is very useful for implementing terrains, quadrics or a lot of other nice things.</li>
<li>Introduce <em>vertex semantics</em>. This is one of the sexiest and most exciting feature of this next
release. I got the idea by thinking about the fact <a href="https://crates.io/crates/luminance">luminance</a> should be low-level but provide
very quick access to building high-level blocks. I got the idea with vertex semantics via my
<a href="https://crates.io/crates/spectra">spectra</a> crate. The idea is simple: currently, we must define a <code>Vertex</code> type and the order
in which the fields appear in the <code>struct</code> defines the bindings the GPU will present to
shaders… but it’s the user responsibility to tell how the shader will fetch those attributes.
This can lead to very wrong situations in which, for instance, the GPU present the <em>normal</em>
attribute as having the index <code>4</code> but the shader tries to fetch them on index <code>2</code>. Vertex
semantics fix this situation by adding an intermediate layer both the user and the GPU (i.e. via
<a href="https://crates.io/crates/luminance">luminance</a>) must speak: semantics. Semantics are user-defined and none is hardcoded in
<a href="https://crates.io/crates/luminance">luminance</a>. As soon as a type uses vertex semantics, all the type system knows how to forward
that information to shaders and what shaders should do to fetch them. The other bonus, ultra
cool thing is that shaders and memory buffers now compose way better: since vertex semantics
define a sort of namespace, you will never have two shaders using the same index for different
semantics (unless you use completely different vertex semantics type). A very cool feature I’m
proud of and that I will detail in a future article / <a href="https://crates.io/crates/luminance">luminance</a> tutorial.</li>
<li>And lots of other doc fixes, enhancements, etc.</li>
</ul>
<h1>One word about the feature set and spare-time</h1>
<p>You might be wondering <em>“Woah, this is such a big feature set. A lot of things are coming! Why not
having split the feature set into several releases?”</em>, and that’s a good question.</p>
<p>Lately, I’ve been feeling on-and-off about everything I do on my spare-time on the FOSS level. I
have several crates I care about and sometimes people show up and start asking for features and/or
bug fixes. This takes some time and after a long day of work, I sometimes feel that I just want to
hack on things I like, play my electric guitar, go to swim or wakeboard or just spend time with
people I care about and love.</p>
<p>Currently, <a href="https://crates.io/crates/luminance">luminance</a> — even though it has an unexpected amount of downloads! — is not a famous
crate. People nowadays talk about crates living in a higher-level stack, such as <a href="https://crates.io/crates/ggez">ggez</a>, <a href="https://crates.io/crates/gfx">gfx</a>,
<a href="https://crates.io/crates/amethyst">amethyst</a>, <a href="https://crates.io/crates/piston">piston</a>, etc. I don’t communicate <em>a lot</em> about <a href="https://crates.io/crates/luminance">luminance</a> because of, mostly, two
things:</p>
<ol>
<li>Imposter syndrom.</li>
<li><em>Avoid success at all cost</em>.</li>
</ol>
<blockquote>
<p>I guess Haskellers will get the second point. ;)</p>
</blockquote>
<p>My own philosophy about software and especially my own software is that it should have a top-tier
quality, both in terms of API design, elegance, type safety, performance and documentation. The
<a href="https://docs.rs/warmy">front documentation of warmy</a> is a good example of something I continuously
seek, because good documentation helps my future myself to read the code and public interface but
also for onboarding people. Lots of projects — especially in professional industries — suffer from
<em>knowledge bubbles</em>, in which just a bunch (often only one!) of people know about <code>A</code> and <code>B</code> and
when someone joins in, they’re lost and have to ask questions. As an engineer, I always write
everything I know in whatever source-of-truth the company I’m in can offer (it can be a Confluence
server; a git repository; a wiki; whatever) because sharing knowledge and writing processes is key
in engineering — also, pro tip: I highly value that when interviewing candidates!</p>
<p>But the more I advance and grow up, the more things I learn and the more things I master. And as I
master new things, I tend to get used to them. And the “mental effort” it requires tends to slowly
disappear. I remember when I tried (hard!) to wrap my finger around what a <em>profunctor</em> is, or what
<em>free monads</em> or even <em>natural transformations</em> are and how to use them. It was hard. As a friend of
mine (<a href="http://dicioccio.fr">@lucasdicciocio</a>) perfectly
summarized it some times ago, when we learn something as a beginner, we struggle. But people who
master those concepts struggle even harder when trying to learn the next concepts at hand. And I’m
not an exception. Today, <em>free monads</em> are something I know (even though I don’t use them; it’s a
myth! it’s a trap! :D) and I feel like I’m not legitimate to talk about them because <em>they feel
easy</em> — and it’s normal, since I’ve learned them. But I see new things to learn, new hard things
that I know I’ll be struggling with for a while. And now it feeds the imposter syndrom. You know,
when you’re 20, you feel like you know a lot of things (and you likely do) and that you’re pretty
confident with your skill base. Today, I’m 27, and I feel like I will always be a rookie, because
there is always something I don’t know to struggle with. And I find that very exciting. Of course,
I don’t project that on others, which is soooo paradoxal: I don’t expect a newcomer to tell me what
a <em>fundep</em> is or know lifetime elision rules in Rust. But I guess I have a very very strict way to
judge my knowledge. I still haven’t figured out if it’s a good thing or if it’s not.</p>
<p>I’ve been told that talking about what I do, what I think, what I know and my engineering philosophy
should help with this kind of questioning. So I decided to use that blog article to share some
thoughts about the next release of <a href="https://crates.io/crates/luminance">luminance</a> that took me so much of my spare-time (but happily!)
and to talk a bit about myself. I know a lot of people in the Rust and Haskell community don’t know
me, but I also know lots of other folks do know me, in both communities. I hope my little vent will
give people some relief — maybe you too have a similar experience? Please share your comments on
<a href="https://www.reddit.com/r/rust/comments/bhslno/pre_luminance100_random_thoughts_about">the Reddit thread this article is post in</a>.</p>
<p>As always, stick doing awesome things and spend your spare-time wisely. Spend time with the ones you
love and do the things you hecking love too!</p>
<p><em>Keep the vibes!</em></p>


guid: https://strongly-typed-thoughts.net/blog/kak-tree-sitter-v0.4Tue, 29 Aug 2023 10:35:00 GMT
title: kak-tree-sitter v0.4 and reflections
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/kak-tree-sitter-v0.4
feedurl: https://phaazon.net/blog/feed
content:
<h1>kak-tree-sitter v0.4.0 announcement and reflections</h1>
<blockquote>
<p>Currently, <a href="https://kakoune.org/">Kakoune</a> doesn’t support writing to files with a timeout, which could help.
<a href="https://github.com/phaazon/kak-tree-sitter/">kak-tree-sitter</a> is a UNIX server bridging <a href="https://kakoune.org/">Kakoune</a> with <a href="https://tree-sitter.github.io/tree-sitter/">tree-sitter</a>, supporting semantic highlighting,
text-objects, indent guidelines and more!</p>
</blockquote>
<p>I want to use the opportunity of releasing <a href="https://github.com/phaazon/kak-tree-sitter/">kak-tree-sitter</a> in version <code>v0.4.0</code> to talk about what’s new but also
explain a bit the new protocol used to make <a href="https://kakoune.org/">Kakoune</a> communicate with the server and vice versa. I think it’s a pretty
fun (and efficient!) way of making things, and it can give people ideas.</p>
<h2>What’s new?</h2>
<p>This new version adds a bunch of things to <a href="https://github.com/phaazon/kak-tree-sitter/">kak-tree-sitter</a>. Among the interesting things:</p>
<ul>
<li>Add support for many languages by default in the shipped <code>config.toml</code>.</li>
<li>Fix PID detection to make it easier to restart / fix the server in case of problems.</li>
<li>Allow to specify the log level of the server.</li>
<li><code>C-c</code> support to stop the server if started as standalone.</li>
<li>Remove <code>tokio</code> and replace with <code>mio</code>. It’s easier, faster and lighter.</li>
<li>And obviously, rewrite the internals to use a new protocol.</li>
</ul>
<p><a href="https://github.com/phaazon/kak-tree-sitter/releases/tag/kak-tree-sitter-v0.4.0">Full changelog here</a>.</p>
<p>Additionnaly, you get <code>v0.4.1</code>, which also brings some interesting changes:</p>
<ul>
<li><code>%opt{kts_lang}</code> is now used to know how to highlight a buffer instead of <code>%opt{filetype}</code> directly. A hook
automatically calls <code>kak-tree-sitter-set-lang</code> (which you can override) to set <code>%opt{kts_lang}</code> before talking to the
server. By default, it simply forwards <code>%opt{filetype}</code>. The goal here is to allow people to provide their own
filetype detection by overriding <code>kak-tree-sitter-set-lang</code>.</li>
<li>Some more internal fixes.</li>
</ul>
<h2>New protocol?</h2>
<p>So yes, the big improvement is the introduction of a new communication protocol between <a href="https://kakoune.org/">Kakoune</a> and <a href="https://github.com/phaazon/kak-tree-sitter/">kak-tree-sitter</a>.
Before <code>v0.4.0</code>, the way things worked was:</p>
<ol>
<li>The user typically starts the server from their <code>kakrc</code>, with a line such as
<code>eval %sh{ kak-tree-sitter --kakoune --daemon --session $kak_session }</code>.</li>
<li>This blocks the editor while the server is starting. Because <code>--daemon</code> is passed, once the server is ready, it’s
going to daemonize so that its main process exits and Kakoune gets control back.</li>
<li>The <code>--kakoune</code> argument does one important thing: it asks the server to send some data back to the editor, once the
server is initialized. The data is sent back via the UNIX socket of Kakoune (<code>kak -p</code>). The session is known via
<code>--session $kak_session</code>.</li>
<li>The data is basically a bunch of options, commands and hooks that automatically call back to the server when a
buffer is opened with a supported filetype. Each command are wrapped in a <code>%sh{}</code> block, and uses the
<code>kak-tree-sitter -r</code> interface to format requests as JSON.</li>
<li>For highlighting requests, the FIFO interface of Kakoune (i.e. <code>kak_command_fifo</code> and <code>kak_response_fifo</code>) is used
to stream buffer content via <code>write</code>.</li>
</ol>
<p>What was the issue with that? Well, all those <code>%sh{}</code> implied to open a shell everytime we wanted to perform a request
to the server. And for highlighting, that happens <em>very often</em> (by default, the trigger hook runs for <code>NormalIdle</code> and
<code>InsertIdle</code>, but you can easily imagine that it runs almost on all keypresses modifying the window or the buffer).
That’s a lot. I quickly came to the realization that I wanted something… faster and with less layers.</p>
<p>See, this <code>kak_command_fifo</code> and <code>kak_response_fifo</code> are pretty interesting FIFO files. The idea is that Kakoune
handles their lifetimes, and they work when you do shell expansions (i.e. <code>%sh{}</code>). You can write to the
<code>$kak_command_fifo</code> Kakoune commands to be executed. This allows to execute Kakoune commands interleaved with shell
commands. You can then use <code>$kak_response_fifo</code> to write back results from Kakoune. I used to write the content of
buffers to <code>$kak_response_fifo</code> and pass the path to this FIFO in the highlight requests so that the server just had
to read from the FIFO and provide the highlights back to Kakoune asynchronously, via <code>kak -p</code>.</p>
<p>FIFOs are wonderful because you can think about them as a 1-1 blocking rendez-vous point between a producer and a
consumer. If a consumer arrives, it blocks until a writer arrives and starts writing, and a writer won’t write until a
consumer arrives. FIFOs do not live on your disk, so writing to them (i.e. <code>write</code> syscall) is super fast and can be
tought as memory-to-memory communication. It’s honestly pretty good.</p>
<p>However, we still have to open a shell to send the requests to the server with this approach… so I needed something
else.</p>
<h2>More FIFOs… MOOOOOORE!</h2>
<p>The idea is actually pretty simple: we can use FIFOs to send requests, in the end. If we have one FIFO for each Kakoune
session, we know that we should only have one writer (i.e. the Kakoune session), and we have a single consumer (the
KTS server). Inside Kakoune, writing to files can be done with the <code>echo -to-file</code> and <code>write</code> command, and we don’t
need a shell for that!</p>
<p>Now the important thing is that Kakoune doesn’t know the path to the FIFO to write requests to, so we still need a
way to get that information. I had two different ideas:</p>
<ol>
<li>We could <em>standardize</em> the place by using the session name. For instance,
<code>$XDG_RUNTIME_DIR/kak-tree-sitter/commands/$kak_session</code>.</li>
<li>Or we could let the server send the path asynchronously, since a Kakoune starting session doesn’t know anything about
the server initially.</li>
</ol>
<p>Currently, this is done with the <code>--kakoune</code> flag. The data that is sent back to the editor contains options
(e.g. <code>%opt{kts_cmd_fifo_path}</code>) so that we know where to write requests.</p>
<blockquote>
<p>In the future, I might actually just standardize the location, since it’s not a random temporary directory.</p>
</blockquote>
<p>The installed hooks, in the <code>kak-tree-sitter</code> group, will contain only a few options, mostly set in the <code>global</code> scope.
The main hook is run whenever a new window is open on a given buffer. A FIFO request is made to KTS with the filetype
(to be more correct, <code>%opt{kts_lang}</code>) and if KTS has the runtime files for this language (grammar, queries, etc.), it
will send back more data to Kakoune, that will be set for the <code>buffer</code> scope. This code will contain the actual
interesting hooks (e.g. <code>NormalIdle</code> and <code>InsertIdle</code>) that will make more FIFO requests to, for instance, highlightthe
buffer.</p>
<p>The content of the buffer is streamed via a FIFO, too — <code>%opt{kts_buf_fifo_path}</code>. Using this pair of FIFOs, we can
send requests to the server and stream buffer content by just using <code>echo -to-file</code> and <code>write</code> Kakoune commands. No
more shell expansions on the hot path!</p>
<blockquote>
<p>Note: currently, buffers must be entirely read by <a href="https://github.com/phaazon/kak-tree-sitter/">kak-tree-sitter</a> because the diffing algorithm is done by
<a href="https://crates.io/crates/tree-sitter-highlight">tree-sitter-highlight</a>, which was great as a PoC, but not ideal now. Partial updates using <code>%val{history}</code> and
<code>%val{uncommitted_modifications}</code> are planned for a future release.</p>
</blockquote>
<p>The only drawback of this approach is that, because we are writing to FIFO files… if one side of the channel is not
there, the other side will block and hang. That could lead to <a href="https://kakoune.org/">Kakoune</a> freezing (for instance, if you kill the server
before trying to quit the editor). However, this is unlikely to happen and is an acceptable tradeoff to me.</p>
<blockquote>
<p>Currently, <a href="https://kakoune.org/">Kakoune</a> doesn’t support writing to files with a timeout, which could help.</p>
</blockquote>
<h2>Conclusion</h2>
<p>This new release is exciting to me. I have learned many things, especially regarding <code>mio</code> vs. <code>tokio</code> (all of those
FIFOs requires a lot of IO synchronization) and optimizing. This new pair of FIFO per session made me realize that,
besides the initial “setup” that requires a shell, we can perfectly communicate with the external world by only writing
to files. In UNIX, everything is, <em>indeed</em>, a file!</p>
<p>The next releases will focus on two main topics:</p>
<ol>
<li>Optimizing even further highlighting, by streaming buffer updates instead of streaming its content. That will
massively increase performance for large buffers.</li>
<li>New features! I plan on adding <strong>theming</strong> (the current situation is weird); <strong>semantic selections</strong> (of course) and…
<strong>indent guidelines</strong>!</li>
</ol>
<p>If you would like to know more about <a href="https://github.com/phaazon/kak-tree-sitter/">kak-tree-sitter</a>, head over
<a href="https://github.com/phaazon/kak-tree-sitter/wiki">to the wiki</a>, which is up to date with <code>v0.4.1</code>.</p>
<p>Have fun and keep the vibes!</p>


title: luminance redesign; Part 1: compatible vertex types
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/2022-luminance-redesign-part-1
feedurl: https://phaazon.net/blog/feed
guid:
https://strongly-typed-thoughts.net/blog/2022-luminance-redesign-part-1Tue, 06 Sep 2022 11:30:00 GMT

content:
<p>Lately, I have decided to take a deep look at <a href="https://crates.io/crates/luminance">luminance</a>. Issues, current code, frustration I had using it, feedback
from people using it, etc. I realized two things:</p>
<ol>
<li>The first commit of <a href="https://crates.io/crates/luminance">luminance</a> was made in 2016, as I migrated it away from its
<a href="https://hackage.haskell.org/package/luminance">Haskell version</a>, which first commit was made in 2015. Since then, my
knowledge of programming, both Haskell and Rust, but also in general has enhanced quite a lot.</li>
<li>The current design has some flaws, both in terms of usability and maintainability.</li>
</ol>
<p>About the first point, I had a look at code that barely changed since I wrote it in 2016, and I’m not completely
satisfied with it anymore. Lots of things happened ever since: I have a sharper vision of how things should be done,
Rust itself has changed quite a lot (back in 2016, Rust 1.0 was just released!), we have more type system features that,
back then, I was lacking, etc.</p>
<p>About the second point, some issues in the current version of <a href="https://crates.io/crates/luminance">luminance</a> (<code>v0.47</code> as writing this blog article) are
complex to solve, such as the <a href="https://github.com/phaazon/luminance-rs/issues/304">Drop problem</a>. Other parts of the API
are insanely complex, like the
<a href="https://docs.rs/luminance/0.47.0/luminance/tess/struct.TessBuilder.html#impl-5"><code>TessBuilder</code></a> related code.</p>
<p>I want all that to change, so I started working on a completely new graphics crate, just to see what kind of interface I
could come up with. And I came up with something much, much easier. So I faced a dilemma: should I release that other
crate and give up on <a href="https://crates.io/crates/luminance">luminance</a>… or should I just port all the new knowledge and ideas from that crate into
<a href="https://crates.io/crates/luminance">luminance</a>?</p>
<p>Well, I have decided to keep <a href="https://crates.io/crates/luminance">luminance</a> around and update it. It is going to take time, because <a href="https://crates.io/crates/luminance">luminance</a> is an
ecosystem of many crates, it has lots of examples (which is great to ensure that its features are still working
correctly once I start redesign it) and because some features are obviously entangled. However, I think it’s worth it,
because even though <a href="https://crates.io/crates/luminance">luminance</a> is not as famous or widely used as <a href="https://crates.io/crates/wgpu">wgpu</a>, it has a special place in the Rust Graphics
community.</p>
<p>So I’m going to blog about the redesign of <a href="https://crates.io/crates/luminance">luminance</a> and it’s going to be a blog article series. I wanted to start
with a very new exciting feature that I have been wanting for a very long time: compatible vertex types.</p>
<h1>The vertex compatibility problem</h1>
<p>When you’re like me and are obsessed with tracking as much information as possible at compile-time, you start imagining
lots of features and ways to <em>prevent</em> people from doing things. Preventing and forbidding is actually much more complex
than allowing something. Think about it like this: assembly allows for everything and is not a complex language. It has
zero constructs, besides opcodes. However, as you add more constructs and abstractions, those abstractions <em>restricts</em>
the application domain; they forbid usage, in a constrained way. There’s a reason why, in Haskell, we use the term
<code>Contraint</code> — which is called, basically, <em>trait bound</em> in Rust.</p>
<p>The way <a href="https://crates.io/crates/luminance">luminance</a> allows a programmer to render objects, which must have a vertex type <code>V</code> that must implement
<code>Vertex</code>. <a href="https://crates.io/crates/luminance-derive">luminance-derive</a> eases the process of implementing lots of traits from <a href="https://crates.io/crates/luminance">luminance</a>. For instance:</p>
<pre><code class="language-rust">#[repr(C)]
#[derive(Clone, Debug, Vertex)] // &lt; notice the Vertex derive here
pub struct MyVertex {
  position: Vector3&lt;f32&gt;,
  normal: Vector3&lt;f32&gt;,
  color: Vector3&lt;u8&gt;,
  weight: Vector3&lt;f32&gt;,
}
</code></pre>
<p>This is great, because the user just has to focus on writing types and just using them. For vertex objects, which can be
imagined as big arrays / buffers of <code>MyVertex</code> here, there are types and functions allowing to basically render such
object, and those render functions expects <code>MyVertex</code>. However, rendering is a multi-stage process. As you may know, the
graphics pipeline contains different steps, and among the first ones, you have a vertex shader, which is responsible in
mapping vertices (to transform them, for instance). In <a href="https://crates.io/crates/luminance">luminance</a>, vertex shaders are strongly-typed with the kind of
vertices they expect. That is to prevent a user from using a vertex shader that expects some vertex attributes that a
vertex object doesn’t have. Imagine that we write a vertex shader that expects a <code>position</code> and a <code>normal</code> but our
vertex object, which uses <code>OtherVertex</code> vertices, only has <code>position</code>. That would result in probably some visual
glitches or just a black screen.</p>
<p>So, we want to have something like this pseudocode:</p>
<pre><code class="language-rust">type VertexShader&lt;V&gt;; // accept only vertex type V
type RenderVertexObjects&lt;W&gt;; // render only vertex type W
</code></pre>
<p>If we want to have a fully typed graphics pipeline, we must have <code>V = W</code>. However, that is going to be very annoying.
Imagine a vertex shader that only needs <code>position</code>, <code>normal</code> and <code>color</code>. Something like:</p>
<pre><code class="language-rust">#[repr(C)]
#[derive(Clone, Debug, Vertex)]
pub struct ShaderVertexInput {
  position: Vector3&lt;f32&gt;,
  normal: Vector3&lt;f32&gt;,
  color: Vector3&lt;u8&gt;,
}
</code></pre>
<p>This is a different type than <code>MyVertex</code>, so if we try to render our <code>MyVertex</code> vertex objects with a vertex shader that
works on <code>ShaderVertexInput</code>, it will get a type mismatch at compile-time. The only way to fix that is to create a copy
of the shader program that works with <code>MyVertex</code> and ignore the <code>weight</code> field. Meh. Waste of resources and duplication.
Everything we don’t want.</p>
<p>When you look at it, the vertex shader is going to use <code>position</code>, <code>normal</code> and <code>color</code>, but it doesn’t have to know
there are other vertex attributes. So it should be able, in theory, to work with <code>MyVertex</code>, since <code>MyVertex</code> has
<code>position</code>, <code>normal</code> and <code>color</code>…</p>
<h1>The solution</h1>
<p>One solution to fix the problem is to introduce a trait, <code>CompatibleVertex</code>, that we will use on the vertex shader code.
The vertex shader still has its <code>ShaderVertexInput</code> type, but it will not require vertex objects to use that type.
Instead, it will require vertex objects to use <code>V</code> and will require <code>ShaderVertexInput: CompatibleVertex&lt;V&gt;</code>. What it
means is that <code>ShaderVertexInput</code> must be <em>included</em> in <code>V</code>. Said otherwise, <code>V</code> must have all of <code>ShaderVertexInput</code>’s
fields (but it can have more).</p>
<pre><code class="language-rust">pub trait CompatibleVertex&lt;V&gt; {}
</code></pre>
<p>Now the big question: how do we implement <code>CompatibleVertex</code> for a given concrete vertex type? Well, in the previous
version of <a href="https://crates.io/crates/luminance">luminance</a>, that used to be implemented manually by users of <a href="https://crates.io/crates/luminance">luminance</a>. It was both unsafe and not really
practical, so I just removed the concept (and <a href="https://crates.io/crates/luminance">luminance</a> in version <code>v0.47</code> and less is subject to the problem I
described above, where you can render a vertex object which vertex type is not compatible with what the shader expects).</p>
<h2>Const generics to the rescue</h2>
<p>Since <code>rustc-1.51</code>, const generics can be used at compile-time. The feature allows to manipulate integers at
compile-time. For instance:</p>
<pre><code class="language-rust">pub struct Array&lt;const N: usize&gt; {
  // …
}
</code></pre>
<p>Here, <code>N</code> is a const generics, and you use it with types like <code>Array&lt;3&gt;</code> or <code>Array&lt;14&gt;</code>. However, that’s currently
(<code>rust-1.63</code>) all you can do with a stable compiler… but we can do much more with a nightly compiler.</p>
<p>There is a feature called <code>adt_const_params</code> that allows using more types, and one type that is very important is
<code>&amp;'static str</code>. Yes, you heard right. Constant strings. We can write something like this:</p>
<pre><code class="language-rust">#![allow(incomplete_features)] // as of rustc-1.65 nightly, this is still required
#![feature(adt_const_params)]

fn hello&lt;const N: &amp;'static str&gt;() {
  println!(&quot;Hello {}!&quot;, N);
}

fn main() {
  hello::&lt;&quot;world&quot;&gt;();
}
</code></pre>
<p>That doesn’t seem like much, but this small change is going to save so many frustration in <a href="https://crates.io/crates/luminance">luminance</a>, and fix our
compatible vertex problems.</p>
<h2>Applying adt_const_params to CompatibleVertex</h2>
<p>What we basically want for one vertex type to compatible with another (i.e. one is <em>included</em> into the other) is to
ensure that the all the fields of one are present in the other. Let’s recall our vertex type definitions and trait:</p>
<pre><code class="language-rust">#[repr(C)]
#[derive(Clone, Debug, Vertex)]
pub struct MyVertex {
  position: Vector3&lt;f32&gt;,
  normal: Vector3&lt;f32&gt;,
  color: Vector3&lt;u8&gt;,
  weight: Vector3&lt;f32&gt;,
}

#[repr(C)]
#[derive(Clone, Debug, Vertex)]
pub struct ShaderVertexInput {
  position: Vector3&lt;f32&gt;,
  normal: Vector3&lt;f32&gt;,
  color: Vector3&lt;u8&gt;,
}

pub trait CompatibleVertex&lt;V&gt; {}
</code></pre>
<p>We can write a <code>HasField</code> trait like this:</p>
<pre><code class="language-rust">pub trait HasField&lt;const NAME: &amp;'static str&gt; {
  type FieldType;
}
</code></pre>
<p>And now, we can implement that trait for all the fields of our vertex types:</p>
<pre><code class="language-rust">// for MyVertex
impl HasField&lt;&quot;position&quot;&gt; for MyVertex {
  type FieldType = Vector3&lt;f32&gt;;
}

impl HasField&lt;&quot;normal&quot;&gt; for MyVertex {
  type FieldType = Vector3&lt;f32&gt;;
}

impl HasField&lt;&quot;color&quot;&gt; for MyVertex {
  type FieldType = Vector3&lt;u8&gt;;
}

impl HasField&lt;&quot;weight&quot;&gt; for MyVertex {
  type FieldType = f32;
}

// for ShaderInputVertex
impl HasField&lt;&quot;position&quot;&gt; for ShaderInputVertex {
  type FieldType = Vector3&lt;f32&gt;;
}

impl HasField&lt;&quot;normal&quot;&gt; for ShaderInputVertex {
  type FieldType = Vector3&lt;f32&gt;;
}

impl HasField&lt;&quot;color&quot;&gt; for ShaderInputVertex {
  type FieldType = Vector3&lt;u8&gt;;
}
</code></pre>
<p>Now, we can write a more meaningful implementor for <code>CompatibleVertex</code> and <code>ShaderInputVertex</code>:</p>
<pre><code class="language-rust">impl&lt;V&gt; CompatibleVertex&lt;V&gt; for ShaderInputVertex where
  V: HasField&lt;&quot;position&quot;, FieldType = Vector3&lt;f32&gt;&gt;
    + HasField&lt;&quot;normal&quot;, FieldType = Vector3&lt;f32&gt;&gt;
    + HasField&lt;&quot;color&quot;, FieldType = Vector3&lt;u8&gt;&gt;
{
}
</code></pre>
<p>Because <code>MyVertex</code> implements the three <code>HasField</code> trait bounds required, it can be used, as well as
<code>ShaderInputVertex</code>.</p>
<p>Obviously, because <a href="https://crates.io/crates/luminance-derive">luminance-derive</a> is a thing, people will never have to write any of those implementors. Actually,
deriving <code>Vertex</code> will do two things (besides what it already does) for a given vertex type:</p>
<ul>
<li>Provide all the <code>HasField</code> implementors for all of its field.</li>
<li>Provide the universal <code>CompatibleVertex</code> implementor.</li>
</ul>
<p>What it means is that, as soon as you start using <code>#[derive(Vertex)]</code>, you will automatically have compatible vertex
types behind the scenes working for you.</p>
<h1>Conclusion</h1>
<p>That feature is already, by itself, worth requiring a nightly compiler, as it brings the exact kind of type safety I
want for my code. And since I started with const generics strings, the next blog article will be about the redesign of
vertex objects (called <code>Tess</code> in the current version of <a href="https://crates.io/crates/luminance">luminance</a>), since, you might have guessed, it will use const
generics strings to upload vertex data per-attribute, and map / update them.</p>
<p>Thanks for having read, and keep the vibe!</p>


guid: https://strongly-typed-thoughts.net/blog/rust-traits-privacyTue, 27 Nov 2018 04:00:00 GMT
title: Rust traits and their (lack of) privacy
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/rust-traits-privacy
feedurl: https://phaazon.net/blog/feed
content:
<h1>On traits privacy interfaces</h1>
<p>I’ve been facing an issue several times in my Rust lifetime experience and never ever have
come up with a pragmatic solution to this problem yet. This problem comes up as follows:</p>
<ul>
<li>Given a trait defined in a library, some code of the library uses its
associated types, methods and/or constraints. <strong>Those items use internal representations
or dependencies that must not appear in the public interface of the library.</strong></li>
<li>That trait is important for the public interface because it gives a practical list of
types that can be used with functions constrained with that trait.</li>
<li>Since we need that trait in the public interface, we <strong>must make it public</strong>.</li>
<li>However, <strong>in Rust, public traits will expose their whole internals to the world.</strong></li>
</ul>
<p>As you can see, we have a situation here. Rust doesn’t allow to expose a trait without
showing its internals. Imagine the following code:</p>
<pre><code>pub fn a_cool_function&lt;T&gt;(id: usize, foo: T) where T: Foo {
  // …
}

pub trait Foo: Sized {
  type Cons: Sized;

  fn compute(self, a: Self::Cons) -&gt; Self;
}

impl Foo for String {
  type Cons = char;

  fn compute(mut self, a: Self::Cons) -&gt; Self {
    self.insert(0, a);
    self
  }
}

impl&lt;T&gt; Foo for VecDeque&lt;T&gt; {
  type Cons = T;

  fn compute(mut self, a: Self::Cons) -&gt; Self {
    self.push_front(a);
    self
  }
}
</code></pre>
<p>We want to export <code>a_cool_function</code>. That function accepts as second argument <em>a type</em> that must
implement the <code>Foo</code> trait. In order for the function not to leak private symbols, <code>Foo</code> then must
be public. However, we don’t want to expose its internals (the <code>Cons</code> associated type nor the
<code>compute</code> method). Why we would want to hide those? I have several points:</p>
<ul>
<li>If you look closely at the implementation, <code>a_cool_function</code> requires its second argument to
implement <code>Foo</code>. Not exposing the internals would then force people to use stock implementors
and will prevent them from providing new ones. This might be wanted for several reasons (unsafe
traits, performances, etc.).</li>
<li>A more sensible reason: imagine that the associated type <code>Cons</code> required a bound on a very
specific trait that is there only because of your implementation details / choices (like a trait
from a dependency). That would leak that trait into the interface, which is not wanted.</li>
<li>As corollary, not exposing the internals of a trait would enable you to change the definition of
the trait without inducing any breaking change, which is an interesting feature.</li>
</ul>
<p>Currently, you can perfectly make a type public without exposing all its methods as public. Why not
having the same power with traits?</p>
<blockquote>
<p>There <em>is</em> a – non-ideal – solution to this problem: <code>#[doc(hidden)]</code> on each items to hide from
the trait. Items tagged with that annotation won’t show up in the documentation, but they will be
definitely usable if a crafty developer reads the source. Not a very good solution to me, thus.</p>
</blockquote>
<h1>Pre-RFC: Enhanced trait privacy</h1>
<p>It’s been a while since I’m looking for a good RFC to introduce this in Rust. This is some needed
jargon, so let’s explain a few terms first:</p>
<ul>
<li>A <em>trait</em> is an open set of types that have common properties, stated by the <em>trait definition</em>.</li>
<li>A <em>trait definition</em> contains associated types, type variables, methods, associated methods
and associated constants.</li>
<li>A <em>bound</em> is found in <code>where</code> clauses to constrain a type or a function.</li>
</ul>
<p>Currently, when you implement <code>Display</code>, you implement the <em>trait</em>. The <code>fmt</code> function you implement
is part of its <em>trait definition</em>. When you write a function like
<code>fn show&lt;T&gt;(x: T) where T: Display</code>, here, <code>Display</code> is not a trait: it’s a <em>bound</em>.</p>
<p><em>Bounds</em> are interesting, because you cannot directly manipulate them. They only appear when
you constrain a function or type. You can combine them, though, with the <code>+</code> operator:</p>
<pre><code>fn borrow&lt;'a, T&gt;(x: &amp;'a T) -&gt; MyBorrow&lt;'a, T&gt; where T: Display + 'a
</code></pre>
<p>This example shows you that lifetimes can be used as bounds as well.</p>
<p>The idea of this RFC is to make a clear distinction between <code>Display</code> as trait and <code>Display</code> as
bound so that it’s possible to use a trait only in bounds position and not implementation. One
major avantage of doing so is to bring completely new semantics to Rust: exposing a trait as
public so that people can pick types that implement the trait without exposing what the trait is
about. This brings a new rule to the game: it’s possible to create ad hoc polymorphism that doesn’t
leak its definition.</p>
<p>The idea is that we love types and we love our type systems. You might come across a situation in
which you need to restrict the set of types that a function can use but in the same time, the
implementation of the trait used to restrict the types is either unsafe, or complex, or depends on
invariants of your crate. In my <a href="https://crates.io/crates/spectra">spectra</a> crate, I have some traits that are currently public that
leak rendering dependencies, which is something I really dislike.</p>
<p>As a <em>prior art</em> section, here’s the wanted feature in Haskell:</p>
<pre><code>{-# LANGUAGE FlexibleInstances #-} -- don’t mind this
{-# LANGUAGE TypeFamilies #-} -- this either

module Lol (
    Foo -- here, we state that we only export the typeclass, not its definition
  ) where

import Data.Text (Text, cons)

class Foo a where
  type Cons a :: *

  compute :: Cons a -&gt; a -&gt; a

instance Foo [a] where
  type Cons [a] = a

  compute = (:)

instance Foo Text where
  type Cons Text = Char

  compute = cons
</code></pre>
<p>Trying to use either the <code>Cons</code> associated type or <code>compute</code> function in a module importing <code>Lol</code>
will result in a compiler error, because those symbols won’t be accessible.</p>
<h2>What it would look like in Rust?</h2>
<p>Currently, there is a <em>weird</em> privacy rule around traits. People not coming from Haskell might feel
okay about that, but I learned Rust years ago while being already fluent with Haskell and got
stunned at this (and I still have microseconds of <em>“Wait, do I not need a <code>pub</code> here? Oh yeah, nah
nah nah.”</em>) When you declare a trait as <code>pub trait …</code>, everything in its definition is automatically
<code>pub</code> as well.</p>
<p>This is so weird because <em>everything else</em> in Rust doesn’t work this way. For instance:</p>
<pre><code>struct Bar; // here, Bar is not pub, so it’s private and scoped to the current module it’s defined in

pub(crate) struct Zoo; // not public either but can be used in other modules of the current crate

pub struct Point { // public
  pub x: f32, // public
  pub y: f32, // public
}

pub struct File { // public
  inode: usize // private
}

enum Either&lt;L, R&gt; { // private
  Left(L), // private
  Right(R), // private
}

pub enum Choice&lt;L, R&gt; { // public
  Left(L), // public (*)
  Right(R), // public (*)
}

// (*): enums require their variants to be public if they’re public for obvious pattern-matching
// exhaustiveness reasons

pub struct Foo; // public

impl Foo {
  fn quux(&amp;self); // not public, only callable in the current module

  pub(crate) fn crab_core_is_funny(self) -&gt; Self; // not public but callable from within this crate

  pub fn taylor_swift() -&gt; Self; // public, callable from the crate and dependent crates
}
</code></pre>
<p>But:</p>
<pre><code>trait PrivTrait { // private trait
  fn method_a(); // private
  fn method_b(); // ditto
  pub fn method_c(); // compilation error and wouldn’t make sense anyway
}

pub trait PubTrait { // public trait
  fn method_a(); // public, even without the pub privacy modifier!!!
  pub(crate) fn method_b(); // won’t compile
  pub fn method_c(); // won’t compile
}
</code></pre>
<p>To me, it would make much more sense for Rust to authorize this:</p>
<pre><code>pub trait PubTrait { // public trait
  fn method_a(); // private, only usable from this module
  pub(crate) fn method_b(); // callable only from modules from this crate
  pub fn method_c(); // public
}
</code></pre>
<p>However, I know, I know. Turning this feature on would break pretty much <strong>everyone’s</strong> code. That’s
why I think – if people are interested by this feature – we should instead go for something like
this:</p>
<pre><code>trait PrivTrait { // private trait
  fn method_a(); // private
  pub(crate) fn method_b(); // compilation error: the trait is private
  pub fn method_c(); // compilation error: the trait is private
}

pub trait PubTrait { // public trait
  fn method_a(); // public (backward compatibility)
  pub(crate) fn method_b(); // callable only from modules from this crate
  pub fn method_c(); // public, akin not to use the pub modifier
  priv fn method_d(); // private; only callable from this module
}
</code></pre>
<p>I’d like to point the reader to <a href="https://github.com/rust-lang/rust/issues/8122">this issue</a>. In its
foreword, <a href="https://github.com/alexcrichton">@alexcrichton</a> rightfully explains that removing the <code>priv</code> keyword was great because it
has yielded a rule ever since – quoting him:</p>
<blockquote>
<p><em>“I think that this would really simplify public/private because there's one and only one rule:
private by default, public if you flag it.”</em></p>
</blockquote>
<p>I feel uncomfortable with the current trait situation because that rule has been broken. <a href="https://github.com/rust-lang/rust/issues/8592">This other
issue</a> pinpoints the problem from another look:
traits shouldn’t set their items visibility based on their own visibilities. This yields weird and
unexpected code rules and precludes interesting design semantics – the one I just described
above.</p>
<p>I hope you liked that article and thoughts of mine. I might write a proper RFC if you peeps are
hyped about the feature – I have personally been wanting this for a while but never found the time
to write about it. Because I care – a lot – about that feature, even more than
<a href="https://phaazon.net/blog/rust-features-documentation">my previous RFC on features discoverability</a>, please feel free to provide constructive criticism,
especially regarding breaking-changes issues.</p>
<p>Keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/on-rust-impl-blockSun, 22 Jul 2018 22:05:00 GMT
title: On impl blocks, injection and surjection in Rust
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/on-rust-impl-block
feedurl: https://phaazon.net/blog/feed
content:
<p>Rust has this cool feature called <em><code>impl</code> block</em>. An <code>impl</code> block is just a scope that introduces a
way to augment a type with methods – do not confuse <code>impl</code> blocks with <em>trait</em> <code>impl</code>s, used to
implement a given trait.</p>
<p>The syntax is the following:</p>
<pre><code class="language-rust">pub struct MyId(u32);

impl MyId {
  /// A method that creates a null identifier.
  pub fn nil() -&gt; Self {
    MyType(0)
  }

  /// A method that checks whether an ID is odd (why not?).
  pub fn is_odd(&amp;self) -&gt; bool {
    self.0 % 2 != 0
  }

  // some other methods…
}
</code></pre>
<p>This syntax gives you the <em>dot notation</em> and the <em>static method</em> call syntax:</p>
<pre><code class="language-rust">let x = MyId::nil(); // static method call

println!(&quot;is it odd? {}&quot;, x.is_odd()); // dot notation
</code></pre>
<p>Most of the time, you’ll want <code>impl</code> blocks for:</p>
<ul>
<li>The <em>dot notation</em>.</li>
<li>Sharing type variables.</li>
</ul>
<h1>Type variables in impl blocks</h1>
<p>A <em>type variable</em> is a variable that holds a type. People coming from languages like C++, Java,
C#, Python, D, etc. are used to name those <em>template parameters</em> or <em>type parameters</em>.</p>
<pre><code class="language-rust">pub struct List&lt;T&gt; {
  // hidden
}

impl&lt;T&gt; List&lt;T&gt; {
  // hidden
}
</code></pre>
<p><code>T</code> here is a type variable, and you can see that you can have an <code>impl</code> block over <code>List&lt;T&gt;</code>. What
that means is that any declared method will have a hidden type <code>Self = List&lt;T&gt;</code>. If you declare a
static method, <code>Self</code> will be set as <code>List&lt;T&gt;</code> and if you declare a regular method, <code>self</code>, <code>&amp;self</code>
or <code>&amp;mut self</code> will have types – respectively – <code>List&lt;T&gt;</code>, <code>&amp;List&lt;T&gt;</code> and <code>&amp;mut List&lt;T&gt;</code>.</p>
<p>That sharing is even more pronounced when you start to deal with more complex data types and want to
restrain their type variables with <em>trait bounds</em>. For instance, imagine that you want an ordered
list:</p>
<pre><code class="language-rust">pub struct OrderedList&lt;T&gt; {
  // hidden
}
</code></pre>
<p>You could implement <code>OrderedList&lt;T&gt;</code> for any <code>T</code>, but it’s very likely that for <em>a lot</em> of methods,
you’ll need <code>T</code> to be comparable. So you can do this, for example:</p>
<pre><code class="language-rust">impl&lt;T&gt; OrderedList&lt;T&gt; where T: PartialOrd {
  // hidden
}
</code></pre>
<p>All the methods and static methods will have <code>Self</code> an alias to <code>List&lt;T&gt; where T: PartialOrd</code>. This
will drastically help you when writing methods that all require the trait bound.</p>
<p>Note that you can have several <code>impl</code> blocks for any kind of refactoring. For instance, if you
<em>also</em> want to have two methods that don’t need the trait bound, you can go and add another <code>impl</code>
block that looks like:</p>
<pre><code class="language-rust">impl&lt;T&gt; OrderedList&lt;T&gt; {
  pub fn new() -&gt; Self {
    // hidden
  }

  pub fn singleton(sole: T) -&gt; Self {
    // hidden
  }
}
</code></pre>
<p>Now let’s try something a bit more complex. Let’s try this:</p>
<pre><code class="language-rust">trait Foo&lt;E&gt; {
  fn foo(&amp;self, x: E);
}

impl&lt;T, E&gt; OrderedList&lt;T&gt; where T: Foo&lt;E&gt; {
  fn invoke_foo(&amp;self, e: E) {
    for t in self {
      t.foo(e);
    }
  }
}
</code></pre>
<p>This won’t work and rustc will complain that <code>E</code> is not contrained enough (it doesn’t appear in
<code>Self</code>, it’s only used in a trait bound on <code>T</code>). So what’s the problem?</p>
<blockquote>
<p>You can test <a href="https://play.rust-lang.org/?gist=993db37f3544929db973861075650158&amp;version=stable&amp;mode=debug&amp;edition=2015">here</a></p>
</blockquote>
<h1>Enter surjection and injection</h1>
<p>The problem with that last block is that <code>E</code> is not constrained on <code>OrderedList&lt;T&gt;</code>. If you
don’t know what it means, don’t freak out: I’m going to introduce every concepts you need to know.
Hang on, it’s gonna be a bit math–ish, but that is worth it.</p>
<h2>Surjection</h2>
<p>In math, a <em>surjection</em> is a property about set morphisms. For a pair of sets <code>C</code> and <code>D</code> and a
morphism <code>f : C -&gt; D</code>, we say that this morphism is <em>surjective</em> if all elements from <code>D</code> appear in
a <code>f</code> application. This is weirdly said, so let’s say it another way: if for all the elements 
<code>y</code> in <code>D</code>, you can find <em>at least one</em> element <code>x</code> in <code>C</code> so that <code>f(x) = y</code>. Another way to
swallow that down if you’re still choking is by drawing the <code>C</code> and <code>D</code> sets as big bags with a few
elements in there. The morphism <code>f</code> applications are just lines from objects from <code>C</code> to <code>D</code>. If all
elements from <code>D</code> have at least one arriving arrow, the morphism is surjective.</p>
<p>We write surjection this way:</p>
<pre><code>Ɐy ∈ D, ∃x ∈ C, f : C -&gt; D / f(x) = y
</code></pre>
<p>And we read it this way:</p>
<ul>
<li><code>Ɐy</code> means <em>for all</em> <code>y</code>.</li>
<li><code>∈ D</code> means <em>is an element of</em> <code>D</code>. So <code>Ɐy ∈ D</code> reads <em>for all y that is an element of D</em>.</li>
<li><code>,</code> often reads as juxtaposition.</li>
<li><code>∃x</code> means <em>there exists</em> <code>x</code>.</li>
<li><code>f : C -&gt; D</code> is just the type definition of the <code>f</code> morphism.</li>
<li><code>/</code> is often used instead of <code>,</code> to delimit the equation and the hypothesis; read it as
<code>so that</code>.</li>
</ul>
<p>So this whole math stuff reads:</p>
<blockquote>
<p>For all <code>y</code> in <code>D</code>, there exists <code>x</code> in <code>C</code> so that <code>f(x) = y</code>.</p>
</blockquote>
<p>It’s quite a non-intuitive notation when you don’t know about it but you can see the concepts are
pretty simple to understand.</p>
<p>You might ask, what is the point of knowing that in our case? Here, <code>Foo</code> is a type-level function.
Rust has decided to go with the <em>mainstream</em> way to note that (which is another topic and should be
the topic of another blog post, to be honest), so yeah, it’s a bit uglier than in Haskell, but it’s
a type-level function. You can picture it as <code>Foo : E -&gt; Foo&lt;E&gt;</code>. You can easily see that if you
are writing the body of a function and that you have a bound like <code>T: Foo&lt;E&gt;</code> available, you know
that you have type <code>T</code> that implements <code>Foo&lt;E&gt;</code>, so there’s definitely one type to substitute <code>E</code>
with (otherwise your function cannot be called), but there could be several ones. This means that
traits are surjective.</p>
<p>Another way to see why it’s surjective: if I give you <code>Foo&lt;u32&gt;</code>, you know that there’s at least one
type implementing it, but you don’t know which one, since it’s ambiguous.</p>
<pre><code class="language-rust">struct A;
struct B;

impl Foo&lt;u32&gt; for A {
  // hidden
}

impl Foo&lt;u32&gt; for B {
  // hidden
}
</code></pre>
<p>If I give you <code>Foo&lt;u32&gt;</code> and tell you <em>“It’s in a bound position of a function”</em>, you know that
if the function gets called, there’s at least one type implementing <code>Foo&lt;u32&gt;</code>… But which one is it?
<code>A</code>? <code>B</code>?</p>
<h2>Injection</h2>
<p>Injection is the <em>reversed</em> version of surjection. It tells you that all the elements from
<code>C</code> have a departing arrow (i.e. morphism application) ending in <code>D</code> <strong>and</strong> the elements in <code>D</code>
have zero or one arriving arrow. Another way to say it is that for all pairs of <code>x</code> and <code>y</code>
in <code>C</code>, <code>f(a) = f(b)</code> implies <code>a = b</code>, which means that you cannot have two inputs mapping to the
same result since if two applications of the morphism yield the same result, it means that the
inputs have to be the same.</p>
<pre><code>Ɐ(x, y) ∈ C, f : C -&gt; D / f(x) = f(y) =&gt; x = y
</code></pre>
<ul>
<li><code>=&gt;</code> is the math notation to state <em>implication</em>.</li>
</ul>
<p>Two interesting properties of injections:</p>
<ul>
<li>Some elements from <code>D</code> (which is called the <em>codomain</em> of the morphism while <code>C</code> is its
<em>domain</em>) might not have any arriving arrow.</li>
<li>If you have an arriving arrow in the codomain, then you can easily reverse its direction and
move backwards to the element in the domain because it’s not possible that two elements map to
the same element.</li>
</ul>
<p>The contrapositive is also interesting:</p>
<pre><code>Ɐ(x, y) ∈ C, f : C -&gt; D /  x ≠ y =&gt; f(x) ≠ f(y)
</code></pre>
<p>It’s easy to imagine an injection in Rust. For instance, the function transforming a boolean into
an integer is an injection. Every value in the domain (<code>bool</code>) have an arrow into the codomain
(<code>u8</code> for instance):</p>
<ul>
<li><code>f(false) = 0</code>.</li>
<li><code>f(true) = 1</code>.</li>
</ul>
<p>You can also see <code>0</code> has a single arriving arrow, and if you take it backwards, you end up on
<code>false</code>. Same thing happens for <code>1</code> and <code>true</code>. Finally, you can see that <code>3</code>, <code>64</code> or <code>42</code>, which
are definitely in the codomain, have no arriving arrow. We are dealing with an injection.</p>
<p>What’s interesting with our specific typesystem problem is that if we had injective traits, that
would mean that given <code>Foo&lt;T&gt;</code>, there’s only one type that can implement this trait. You would then
be able to take the bound backwards and recover the type, removing the ambiguity.</p>
<blockquote>
<p>This is not correctly possible in Rust. If you’re interested, Haskell can do it via
<a href="https://ghc.haskell.org/trac/ghc/wiki/InjectiveTypeFamilies">injective type families</a> or
functional dependencies, for instance.</p>
</blockquote>
<h3>A small digression on bijection</h3>
<p>For curiosity only, bijection is just the superposition of both the properties. If you have a
morphism that is both surjective and injective, it is bijective, and any element in the domain
gives you one element in the codomain and any element in the codomain gives you one element in the
domain an element in the codomain must have exactly one arriving arrow.</p>
<h1>Back to our impl block problem</h1>
<pre><code class="language-rust">trait Foo&lt;E&gt; {
  fn foo(&amp;self, x: E);
}

impl&lt;T, E&gt; OrderedList&lt;T&gt; where T: Foo&lt;E&gt; {
  fn invoke_foo(&amp;self, e: E) {
    for t in self {
      t.foo(e);
    }
  }
}
</code></pre>
<p>This is our initial problem. As you can see here, the <code>impl</code> block has two variables: <code>T</code> and <code>E</code>.
<code>T</code> here means that we’re implementing methods <em>for all</em> the <code>T</code>s – i.e. <code>Ɐ</code>. However, we’re not
implementing it for all the <code>E</code>. We would like to state that <em>we need a <code>E</code> to exist</em>, which means
that <em>there exists</em> an <code>E</code> – i.e. <code>∃</code>. The current syntax doesn’t support this.</p>
<p>However, what happens if we do this:</p>
<pre><code class="language-rust">impl&lt;T&gt; OrderedList&lt;T&gt; {
  fn invoke_foo&lt;E&gt;(&amp;self, e: E) where T: Foo&lt;E&gt; {
    for t in self {
      t.foo(e);
    }
  }
}
</code></pre>
<p>Here, you can see that since we can provide <code>E</code> by the caller of the <code>invoke_foo</code> function, we don’t
require an injective bound: we narrow all the possible types to one provided by the caller of the
function.</p>
<blockquote>
<p>You can <a href="https://play.rust-lang.org/?gist=d3c84c52f211736cff956c22ba72a741&amp;version=stable&amp;mode=debug&amp;edition=2015">test it by yourself here</a>.
The <code>Clone</code> trait was added so that we can actually loop with the input argument.</p>
</blockquote>
<h1>Intuition</h1>
<p>All of this made me think about why do we even use <code>impl</code> blocks. I mean, Haskell has been around
for roughly 30 years and we never seen the need for it arise. Rust has a very special way to express
typeclasses (Rust’s traits have an implicit type parameter, <code>Self</code>, which is a sort of limitation
that might get fixed in the future). If we forget about the <em>dot notation</em>, I truly think that I
would drop <code>impl</code> blocks for several reasons:</p>
<ul>
<li>No solution currently exists to the problem this blog entry discusses with <code>impl</code> blocks so far.</li>
<li>Documentation might become very hard to read because it makes every methods dependent on an
<code>impl</code> block in the documentation. This refactoring might be interesting for code readers and
writers but is, to me, truly a nightmare to people reading the documentation – it kind of makes
reading the documentation <em>stateful</em>, which is ridiculous. However, this point might be a very
interesting <code>rustdoc</code> feature request! ;)</li>
<li>On a general note, I’m not really adding any constraint to the <code>impl</code> blocks because those
depend mostly on the functions / methods and I find it utterly stupid to create <code>n</code> blocks for
<code>n</code> functions. So I basically just spawn the most general form of <code>impl</code> block and put the 
constraints next to the methods.</li>
<li>I often confuse them with <code>impl</code> trait implementors when reading the code from someone else.</li>
</ul>
<p>What would be really great would be to allow people to create function with a <code>self</code> argument
<em>without <code>impl</code> block</em>. You could still have them around for compatibility purposes and for people
who actually enjoy them, but to me those are a bit useless and boilerplate.</p>
<blockquote>
<p>Again, I’m only talking about <code>impl</code> blocks. <code>impl</code> for trait are actually quite nice since you
can see them and spot them easily (as the <code>instance</code> keyword in Haskell).</p>
</blockquote>
<p>That’s all for today. Thank you for having read me. Keep the vibes, and write RFCs!</p>


guid: https://strongly-typed-thoughts.net/blog/auto_derive_luminance-0.25.5Thu, 01 Mar 2018 02:18:00 GMT
title: Auto-derive uniform interfaces in luminance-0.25.5
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/auto_derive_luminance-0.25.5
feedurl: https://phaazon.net/blog/feed
content:
<p><a href="https://crates.io/crates/luminance">luminance</a> was released in <a href="https://crates.io/crates/luminance/0.25.5">version 0.25.5</a> two weeks ago and I realized I haven’t blogged about it
while it’s an important release!</p>
<blockquote>
<p>Actually, it’s from 0.25.2, but I’ll just talk about the latest patched.</p>
</blockquote>
<p>That new release received several patches:</p>
<ul>
<li>The <code>gl</code> dependencies was updated to <a href="https://crates.io/crates/gl/0.10.0">gl-0.10.0</a>.</li>
<li>A new <code>uniform_interface!</code> macro was added.</li>
</ul>
<p>I’ll introce that <code>uniform_interface!</code> macro in this blog entry.</p>
<h1>Auto derive <strike>free</strike> <em>almost free</em></h1>
<p>That macro was written to provide a smoother and better experience when writing types that will act
as <em>uniform interfaces</em> in shader. For the record, when you create a <code>Program&lt;_, _, _&gt;</code> with
<a href="https://crates.io/crates/luminance">luminance</a>, the third type variable is called the <em>uniform interface</em>. You’ll be handed back a
value of that type when your shader is being used. That will give you the possibility to send data
to the sader prior to making any rendering.</p>
<p>The main idea is to have a type like this:</p>
<pre><code>struct Common {
  resolution: Uniform&lt;[f32; 2]&gt;,
  time: Uniform&lt;f32&gt;,
  jitter: Uniform&lt;f32&gt;
}
</code></pre>
<p>If you build a program which type is <code>Program&lt;_, _, Common&gt;</code>, whenever you ask a pipeline to use
that program, you’ll be handed a <code>Common</code> object so that you can access back the <em>uniforms</em>.</p>
<p>In order for that to happen, though, there’s a constraint: your type must implement a trait. That
trait is <a href="https://docs.rs/luminance/0.25.5/luminance/shader/program/trait.UniformInterface.html"><code>UniformInterface</code></a>. There’s no magic behind that – though, it can be a bit harsh to
implement that trait. If you’re using <a href="https://crates.io/crates/luminance">luminance</a> or plan to, I strongly advise you to try an
<code>impl</code> that trait by hand first.</p>
<p>Anyway, after the fifth or sixth <em>uniform interface</em> you’ll have written, you’ll start to get bored
of writing the same code every now and then. That’s the exact same feeling as for <em>serializing</em> and
<em>deserializing</em> with <a href="https://crates.io/crates/serde">serde</a>: it’s interesting to do it by hand the first time, but it gets really,
<strong>really</strong> annoying after a while.</p>
<p>So it’d be great to be able to automatically <em>derive</em> <code>UniformInterface</code> for your own types.
There’re two solutions here:</p>
<ol>
<li>Use <a href="https://doc.rust-lang.org/book/first-edition/procedural-macros.html">procedural macros</a> so that we can write something like <code>#[derive(UniformInterface)]</code>.</li>
<li>Use standard <code>macro_rules!</code>.</li>
</ol>
<p>(1) is great because it seems <em>seamless</em> when you use it – you already use
<code>#[derive(Clone, Debug, Eq, PartialEq, Etc)]</code>. However, even though it’s doable to learn <a href="https://crates.io/crates/syn">syn</a> in
order to parse the Rust token stream, it’ll require some time and patience while I wanted something
quick to mockup the idea. For that, (2) is perfect, because regular macros are <em>easy</em> and if
correctly written, shouldn’t add too much weirdness over the type.</p>
<blockquote>
<p>The idea is to move to (1) if I find (2) to be a success and really useful.</p>
</blockquote>
<h1>The macro</h1>
<p>So… we’re talking about the <a href="https://docs.rs/luminance/0.25.5/luminance/macro.uniform_interface.html"><code>uniform_interface!</code></a> macro. This macro has already a pretty decent
documentation, so feel free to visit it, but I’ll explain more here.</p>
<p>A macro is generally used to introduce an <a href="https://wiki.haskell.org/Embedded_domain_specific_language">EDSL</a>. For instance, <a href="https://crates.io/crates/nom">nom</a>’s <a href="https://docs.rs/nom/3.2.1/nom/macro.do_parse.html"><code>do_parse</code></a> macro
uses a funny EDSL (you use operators like <code>&gt;&gt;</code> and return result with <code>(…)</code>, which is not
<em>normal</em> Rust code).</p>
<p>The <a href="https://docs.rs/luminance/0.25.5/luminance/macro.uniform_interface.html"><code>uniform_interface!</code></a> macro uses an EDSL that you know very well: it’s plain Rust code! Hot
news: you’ll have nothing more to learn!</p>
<blockquote>
<p>Note: it’s not really <em>any</em> Rust code, since you can only define <code>struct</code>s. Then, see that as a
subset of Rust.</p>
</blockquote>
<p>The syntax is very simple. Let’s take back our sample from above with the <code>Common</code> interface but now
let’s the macro do all the magic for us!</p>
<pre><code>#[macro_use]
extern crate luminance;

uniform_interface! {
  struct Common {
    resolution: [f32; 2],
    time: f32,
    jitter: f32
  }
}
</code></pre>
<p>You’ll notice two interesting properties:</p>
<ul>
<li>You define your fields without annotating them with <code>Uniform&lt;_&gt;</code>: the macro does it for you.</li>
<li>The <code>UniformInterface</code> <code>impl</code> is automatically generated for you!</li>
</ul>
<p>The second point is the most interesting one, since it’s akin to writing
<code>#[derive(UniformInterface)]</code>. As you can see, the syntax overhead is not really a problem.</p>
<p>The cool part of that macro is that it also supports Rust annotations on its fields. You have two
possible annotations available:</p>
<ul>
<li><code>as(&quot;something&quot;)</code> behaves by not using the field’s direct name and instead use the one provided as
argument.</li>
<li><code>unbound</code> enables not to make the whole <em>uniform interface</em> fail if a field cannot be mapped on
the GPU side.</li>
</ul>
<p>The <code>as(…)</code> annotation is very simple and straight-forward to get, since it lets you change the
binding name of the field. The <code>unbound</code> annotation is a bit trickier and you need to understand how
<code>UniformInterface</code> expects the value to be constructed.</p>
<p>By default, when you try to map a field to something on the GPU side (shader program), you use the
<a href="https://docs.rs/luminance/0.25.5/luminance/shader/program/struct.UniformBuilder.html#method.ask"><code>UniformBuilder::ask</code></a> function. If you have a closer look at function, you can see that it returns
<code>Result&lt;Uniform&lt;T&gt;, UniformWarning&gt;</code>. In order to get the <code>Uniform&lt;T&gt;</code> out, you’re left with the
choice of how you should handle errors. And that will depend on how you want your value to be built.
For instance, some fields might be completely mandatory for your shader to work and others optional.
<code>unbound</code> tags a field as optional.</p>
<p>You can mix both annotations if you want to remap an optional field!</p>
<pre><code>#[macro_use]
extern crate luminance;

uniform_interface! {
  // we want to export that, so we also make it pub
  pub struct Common {
    resolution: [f32; 2], // this is really required
    #[as(&quot;t&quot;)]
    time: f32, // will be referenced with &quot;t&quot; in the shader and is mandatory as well
    #[unbound, as(&quot;bias&quot;)]
    jitter: f32 // will be referenced with &quot;bias&quot; in the shader and is optional
  }
}
</code></pre>
<p>That’s all for today! I hope that feature will be useful for whomever uses <a href="https://crates.io/crates/luminance">luminance</a> – I use it
extensively and I’ve been using <code>uniform_interface!</code> for several days now and it’s <em>really</em>
appreciated! :D</p>
<blockquote>
<p>I plan to add another macro to create <em>buffer types</em>, since they must meet GPU-specific alignment
rules, who are boring to maintain without code generation.</p>
</blockquote>
<p>Feel free to provide feedback on Reddit or GitHub and have fun!</p>


guid: https://strongly-typed-thoughts.net/blog/volumetric_light_shaftsWed, 08 Apr 2015 00:00:00 GMT
title: Volumetric light shafts
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/volumetric_light_shafts
feedurl: https://phaazon.net/blog/feed
content:
<h1>Volumetric light shafts</h1>
<p>On the last weekend, I was at the Revision demoparty (Easterparty). It’s a
meeting of friends and great people doing graphics, music, code and that kind
of fun and pretty stuff. I didn’t release anything, but I’ve been working on
a release for a while now, and I was lacking something in my engine: <em>light 
shafts</em>.</p>
<p>I know a few ways to do that. They almost all fall in one of the two following
classes:</p>
<ol>
<li>screen-space-based;</li>
<li>raymarching-based.</li>
</ol>
<p>Both the techniques produce interesting images. However, the <em>screen-space-based</em>
method produces bad results when you don’t look directly to the light – it may
even produce nothing if the light is out of screen – and the <em>raymarching-based</em>
is a step process that might generate artifacts and can be slow.</p>
<p>The idea I had is <strong>very simple</strong>. I haven’t tested it yet, but it’s planned for
very soon. I’ll post images with benchmarks as soon as I have something on screen.
I’m not sure it’s an unknown way to do it. I just haven’t found anything
describing that yet. If you have, please leave a link in the comments below! :)</p>
<h1>Volume extraction</h1>
<p>The idea is the following. You need a depthmap. If you’re used to <em>shadow mapping</em>
you already have a depthmap for your light. In order to simplify this, we’ll use the
depthmap used for shadow mapping, but I guess it’s totally okay to use another
depthmap – we’ll see that it could be even handier.</p>
<p>For each point in that depthmap is the distance – in a specific space coordinates
system – of the corresponding point in world space to the 
position of the light. If you have the projection and the view matrices of the
light, it’s easy to deproject the depthmap. What would we get if we deproject
<strong>all the depthmap texels</strong> into the world space? We’d get the exact lit surfaces.</p>
<p>For a spot light, you can imagine the deprojected version of the depthmap as a cone
of light. The “disk” of the cone will deform and shape as the lit environment. That’s
the first part of the algorithm.</p>
<p>We have a points cloud. What happens if, for each deprojected texel – i.e. point – we
draw a line to the position of the light? We get an actual lines field representing…
photons paths! How amazing is that?! Furthermore, because of the depthmap sampling,
if you look at the light and that an object is between you and the light, the photons
paths won’t go through the obstructing object! Like the following image:</p>
<p><img src="http://i280.photobucket.com/albums/kk167/Bea_Douglas/ShaftsofLightMountBaker-SnoqualmieN.jpg" alt="" /></p>
<p>Of course, because of using raw lines, the render might be a bit blocky at first. If you
know the <em>laser trick</em>[^1] – i.e. quad-based lasers, you can apply it to our lines as
well, in order to get better results. The first improvement is to disable depth
test and enable additive blending.</p>
<h1>Algorithm</h1>
<p>In the first place, you need to generate the depthmap of the light. Then, you need to extract
the volumetric shaft. You’ll need a vertex buffer for that. Allocate <strong>w*h</strong> elements, where
<em>w</em> and <em>h</em> are the depthmap’s <em>width</em> and <em>height</em>. Yeah, a point per texel.</p>
<p>Create a shader and make a render with <code>glDrawArrays(GL_POINTS, 0, w*h)</code> with an <em>attributeless
vertex array object</em>. Don’t forget to bind the depthmap for use in the shader.</p>
<p>In your vertex shader, use <code>gl_VertexID</code> to sample from the depthmap. Transform the resulting
texel in world-space. You can use something like the following deprojection formula:</p>
<pre><code class="language-C">vec3 deproject() {
  float depth = 2. * texelFetch(depthmap, ivec2(gl_FragCoord.xy), 0).r - 1.;
  vec4 position = vec4(vv, depth, 1.);
  position = iProjView * position;
  position.xyz /= position.w;
  return position.xyz;
}
</code></pre>
<p>Pass that to the next stage, the geometry shader. There, build whatever kind of new primitive
you want. In the first place, I’ll go for a simple line connected to the light’s position. In
further implementation, I’ll go for lasers-like base shapes, like star-crossed quads.</p>
<p>In the fragment shader, put whatever color you want the shaft to have. You could use 
interpolation to reduce lighting wherever you want to create nice effects.</p>
<p>Don’t forget to use additive blending, as we do for lasers.</p>
<h1>Considerations</h1>
<p>I see two major problems. The first one is the bright aspect the shaft will have if you don’t
blend correctly. Play with alpha to reduce more if the fragment is near the light’s position and
make the alpha bigger when you’re far away from the light’s position. Because you’ll blend way more
photons paths near the light’s position than far from it.</p>
<p>The second issue is the resolution of the extracted volume. For a 512x512 depthmap, you’ll get
around 262k points, then 524k lines. That’s a lot for such an object. And that’s only for a simple
spot light. An omnidirectional light would require six times more lines. What happens if we don’t
use lines, but star-crossed quads an that we want several shafts? You see the problem.</p>
<p>A solution could be to sample from high mipmap level, so that you don’t use the full resolution of
the shadow map. That would result in less visual appealing shafts, but I’m pretty sure it’d be still
good. You could also branch a blur shader to smooth out the whole thing.</p>
<h1>Conclusion</h1>
<p>I’ll try to implement that as soon as possible, because I think my idea is pretty interesting compared
to raymarching, which is expensive, and way better than screen-space, because the shaft will still be
visible if the light goes out of screen.</p>
<p>[^1]: I’ll write an article about it if you don’t – leave a comment for asking</p>


guid: https://strongly-typed-thoughts.net/blog/workflowSun, 23 Jul 2017 00:00:00 GMT
title: On programming workflows
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/workflow
feedurl: https://phaazon.net/blog/feed
content:
<p>For the last month, I had technical talks with plenty of programmers from all around the globe – thank
you IRC! Something interesting that often showed up in the discussion was the actual <em>workflow</em> we have
when writing code. Some people are use to <a href="https://en.wikipedia.org/wiki/Integrated_development_environment">IDE</a> and won’t change their tools for anything else. Some
others use very basic text editors (I even know someone using <a href="https://www.barebones.com/products/bbedit">BBEdit</a> for his job work! crazy,
isn’t it?). I think it’s always a good thing to discuss that kind of topic, because it might give
you more hindsight on your own workflow, improve it, share it or just show your nice color scheme.</p>
<p>I’ll be talking about:</p>
<h1>My editor</h1>
<p>I’ve tried a lot of editors in the last ten years. I spent a year using emacs but eventually
discovered – erm, learned! – vim and completely fell in love with the <em>modal</em> editor. It was
something like a bit less than ten years ago. I then tried a lot of other editors (because of
curiosity) but failed to find something that would help be better than vim to write code. I don’t
want to start an editor war; here’s just my very personal point of view on editing. The concept of
modes in vim enables me to use a very few keystrokes to perform what I want to do (moving, commands,
etc.) and I feel very productive that way.</p>
<blockquote>
<p>A year ago, a friend advised me to switch to <a href="https://neovim.io">neovim</a>, which I have. My editor of the time is then
neovim, but it’s so close to vim that I tend to use (neo)vim. :)</p>
</blockquote>
<p>I don’t use any other editing tool. I even use neovim for taking notes while in a meeting or when I
need to format something in Markdown. I just use it everywhere.</p>
<h1>My neovim setup</h1>
<p>I use several plugins:</p>
<pre><code>Plugin 'VundleVim/Vundle.vim'
Plugin 'ctrlpvim/ctrlp.vim'
Plugin 'gruvbox'
Plugin 'neovimhaskell/haskell-vim'
Plugin 'itchyny/lightline.vim'
Plugin 'rust-lang/rust.vim'
Plugin 'jacoborus/tender.vim'
Plugin 'airblade/vim-gitgutter'
Plugin 'tikhomirov/vim-glsl'
Plugin 'plasticboy/vim-markdown'
Plugin 'cespare/vim-toml'
Plugin 'mzlogin/vim-markdown-toc'
Plugin 'ElmCast/elm-vim'
Plugin 'raichoo/purescript-vim'
Plugin 'easymotion'
Plugin 'scrooloose/nerdtree'
Plugin 'ryanoasis/vim-devicons'
Plugin 'tiagofumo/vim-nerdtree-syntax-highlight'
Plugin 'mhinz/vim-startify'
Plugin 'Xuyuanp/nerdtree-git-plugin'
Plugin 'tpope/vim-fugitive'
Plugin 'MattesGroeger/vim-bookmarks'
Plugin 'luochen1990/rainbow'
</code></pre>
<h2>Vundle</h2>
<p>A package manager. It just takes the list you read above and clones / keeps updated the git
repositories of the given vim packages. It’s a must have. There’re alternatives like Pathogen but
Vundle is very simple to setup and you don’t have to care about the file system: it cares for you.</p>
<h2>ctrlp</h2>
<p>This is one is a must-have. It gives you a fuzzy search buffer that traverse files, MRU, tags,
bookmarks, etc.</p>
<p><img src="http://phaazon.net/pub/ctrlp.png" alt="" /></p>
<p>I mapped the file search to the <code>, f</code> keys combination and the tag fuzzy search to <code>, t</code>.</p>
<h2>gruvbox</h2>
<p>The colorscheme I use. I don’t put an image here since you can find several examples online.</p>
<blockquote>
<p>This colorscheme also exists for a lot of other applications, among terminals and window managers.</p>
</blockquote>
<h2>haskell-vim</h2>
<p>Because I write a lot of Haskell, I need that plugin for language hilighting and linters… mostly.</p>
<h2>lightline</h2>
<p>The <a href="https://github.com/itchyny/lightline.vim">Lightline</a> vim statusline. A popular alternative is <a href="https://github.com/powerline/powerline">Powerline</a> for instance. I like Lightline
because it’s lightweight and has everything I want.</p>
<h2>rust.vim</h2>
<p>Same as for Haskell: I wrote a lot of Rust code so I need the language support in vim.</p>
<h2>tender</h2>
<p>A colorscheme for statusline.</p>
<h2>vim-gitgutter</h2>
<p>I highly recommend this one. It gives you diff as icons in the symbols list (behind the line numbers).</p>
<p><img src="http://phaazon.net/pub/vim-gitgutter.png" alt="" /></p>
<h2>vim-glsl</h2>
<p>GLSL support in vim.</p>
<h2>vim-markdown</h2>
<p>Markdown support in vim.</p>
<h2>vim-toml</h2>
<p>TOML support in vim.</p>
<blockquote>
<p>TOML is used in Cargo.toml, the configuration file for Rust projects.</p>
</blockquote>
<h2>vim-markdown-toc</h2>
<p>You’re gonna love this one. It enables you to insert a table of contents wherever you want in a
Markdown document. As soon as you save the buffer, the plugin will automatically refresh the TOC for
you, keeping it up-to-date. A must have if you want table of contents in your specifications or RFC
documents.</p>
<h2>elm-vim</h2>
<p>Elm support in vim.</p>
<h2>purescript-vim</h2>
<p>Purescript support in vim.</p>
<h2>easymotion</h2>
<p>A must have. As soon as you hit the corresponding keys, it will replace all words in your visible
buffer by a set of letters (typically one or two), enabling you to just press the associated
characters to jump to that word. This is the <em>vim motion</em> on steroids.</p>
<h2>nerdtree</h2>
<p>A file browser that appears at the left part of the current buffer you’re in. I use it to discover
files in a file tree I don’t know – I use it very often at work when I work on a legacy project, for
instance.</p>
<h2>vim-devicons</h2>
<p>A neat package that adds icons to vim and several plugins (nerdtree, ctrlp, etc.). It’s not a must-have
but I find it gorgeous so… just install it! :)</p>
<h2>vim-nerdtree-syntax-highlight</h2>
<p>Add more formats and files support to nerdtree.</p>
<h2>vim-startify</h2>
<p>A <em>cute</em> plugin that will turn the start page of vim into a MRU page, enabling you to jump to the
given file by just pressing its associated number.</p>
<blockquote>
<p>It also features a cow that gives you fortune catchphrases. Me gusta.</p>
</blockquote>
<h2>nerdtree-git-plugin</h2>
<p>Git support in nerdtree, it adds markers in front of file that have changes, diff, that were added,
etc.</p>
<h2>vim-fugitive</h2>
<p>A good Git integration package to vim. I use it mostly for its <code>Gdiff</code> diff tooling directly in vim,
even though I like using the command line directly for that. The best feature of that plugin is
the integrated blaming function, giving you the author of each line directly in a read-only vim
buffer.</p>
<h2>vim-bookmarks</h2>
<p>Marks on steroids.</p>
<h2>rainbow</h2>
<p>This little plugins is very neats as it allows me to add colors to matching symbols so that I can
easily see where I am.</p>
<p><img src="http://phaazon.net/pub/rainbow_vim.png" alt="" /></p>
<h1>Workflow in Rust</h1>
<p>I’m a rustacean. I do a lot of Rust on my spare time. My typical workflow is the following:</p>
<ol>
<li>I edit code in neovim</li>
<li>Depending on the project (whether I have a robust unit tests base or not or whether I’m writing a
library or an app), I use several <code>cargo</code> commands:
<ul>
<li>for a library project, I split my screen in two and have a <code>cargo watch -x test</code> running; this
command is a file watcher that will run all the tests suites in the project as soon as a file
changes;</li>
<li>for an app project, I split my screen in two and have a <code>cargo watch</code> – similar to <code>cargo watch -x check</code>
running; this command is a file watcher that will proceed through the compilation of the binary
but it will not compile it down to an actual binary; it’s just a check, it doesn’t produce
anything you can run. You can manually run that command with <code>cargo check</code>. See more
<a href="https://github.com/rust-lang/cargo/pull/3296#event-893283611">here</a>;</li>
<li>for other use cases, I tend to run <code>cargo check</code> by hand and run <code>cargo build</code> to test the
actualy binary</li>
</ul>
</li>
<li>Because I’m a unix lover, I couldn’t work correctly without a terminal, so I use neovim in a
terminal and switch from the console to the editor with the keybinding <code>C-z</code> and the <code>jobs</code> and
<code>fg</code> commands. I do that especially to create directories, issue git commands, deploy things,
etc. It’s especially ultra cool to run a <code>rg</code> search or even a <code>find</code> / <code>fd</code>. I sometimes do that
directly in a neovim buffer – with the <code>:r!</code> command – when I know I need to edit things from the
results. Bonus: refactoring with <code>tr</code>, <code>sed</code> and <code>find -exec</code> is <strong>ultra</strong> neat.</li>
</ol>
<h1>Workflow in Haskell</h1>
<p>The workflow is almost exactly the same besides the fact I use the <code>stack build --fast --file-watch</code>
command to have a file watcher.</p>
<blockquote>
<p>Haskell’s stack doesn’t currently the awesome <code>check</code> command Rust’s cargo has. Duh :(</p>
</blockquote>
<p>I also have a similar workflow for other languages I work in, like Elm, even though I use standard
unix tools for the file watching process.</p>
<h1>Git workflow</h1>
<p>Aaaah… <code>git</code>. What could I do without it? Pretty much nothing. <code>git</code> is such an important piece of
software and brings such an important project philosophy and behaviors to adopt.</p>
<p>What I find very cool in git – besides the tool itself – is everything around it. For instance, on
GitHub, you have the concept of Pull Request (PR) – Merge Request in GitLab (MR). Associated with a
good set of options, like disallowing people to push on the <code>master</code> branch, hooks, forcing people
to address any comments on their MR, this allows for better code reviews and overall a way better
quality assurance of the produced code than you could have without using all this infrastructure.
Add a good set of DevOps for deployment and production relatide issues and you have a working team
that has no excuse to produce bad code!</p>
<h2>Some git candies I love working with</h2>
<p>My neovim fugitive plugin allows me to open a special buffer with the <code>:Gblame</code> command that gives
me a <code>git blame</code> annotation of the file. This might be trivial but it’s very useful, especially at
work when I have my colleagues next me – it’s always better to directly ask them about something
than guessing.</p>
<p><img src="http://phaazon.net/pub/vim_fugitive_blame.png" alt="" /></p>
<p>Another one that I love is <code>:Gdiff</code>, that gives you a <code>git diff</code> of the modification you’re about to
stage. I often directly to a <code>git diff</code> in my terminal, but I also like how this plugin nails it as
well. Very pratictal!</p>
<p><img src="http://phaazon.net/pub/vim_fugitive_diff.png" alt="" /></p>
<h1>General note on workflow</h1>
<p>It’s always funny to actually witness difference in workflows, especially at work. People who use
mostly IDEs are completely overwhelmed by my workflow. I was completely astonished at work that some
people hadn’t even heard of <code>sed</code> before – they even made a Google search! I’m a supporter of the
philosophy that one should use the tool they feel comfortable with and that there’s no “ultimate”
tool for everyone. However, for my very own person, I really can’t stand IDEs, with all the buttons
and required clicks you have to perform all over the screen. I really think it’s a waste of time,
while using a modal editor like neovim with a bépo keyboard layout (French dvorak) and going back
and forth to the terminal is just incredibly simple, yet powerful.</p>
<p>I had a pretty good experience with <a href="https://atom.io">Atom</a>, a modern editor. But when I’ve been told it’s written
with web technologies, the fact it’s slow as f<em>ck as soon as you start having your own tooling
(extensions), its pretty bad behavior and incomprensible “Hey, I do whatever the f</em>ck I want and
I’ll just reset your precious keybindings!” or all the weird bugs – some of my extensions won’t just
work if I have an empty pane open, wtf?!… well, I was completely bolstered that GUI interfaces, at
least for coding and being productive, are cleary not for me. With my current setup, my hands
<em>never</em> move from the keyboard – my thrists are completely static. With all the candies like
easymotion, ctrlp, etc. etc. I just can’t find any other setups faster and comfier than this one.</p>
<p>There’s even an extra bonus to my setup: because I use mostly unix tools and neovim, it’s pretty
straigth-forward to remote-edit something via ssh, because everything happens in a terminal. That’s
not something you can do easily with Atom, Sublime Text or any other editors / IDEs – and you even
pay for that shit! No offence!</p>
<p>However, there’s a caveat: because pretty much everything I do is linked to my terminal, the user
experience mostly relies on the performance of the terminal. Using a bad terminal will result in an
overall pretty bad experience, should it be editing, compiling, git or ssh. That’s why I keep
lurking at new terminal emulaters – <a href="https://github.com/jwilm/alacritty">alacritty</a> seems very promising, yet it’s still too buggy and
lacks too many features to be production-ready to me – but it’s written in Rust and is
GPU-accelerated, hells yeah!</p>
<h1>Conclusion</h1>
<p>Whoever you are, whatever you do, whomever you work with, I think the most important thing about
workflow is to find the one that fits your needs the most. I have a profound, deep digust for
proprietary and closed software like Sublime Text and IDEs that use GUIs while keyboard shortcut are
just better. To me, the problem is about the learning curve and actually wanting to pass it –
because yes, learning (neo)vim in details and mastering all its nits is not something you’ll do in
two weeks; it might take months or years, but it’s worth it. However, as I said, if you just feel
good with your IDE, I will not try to convert you to a modal editor or a unix-based workflow…
because you wouldn’t be as productive as you already are.</p>
<p>Keep the vibe!</p>


guid: https://strongly-typed-thoughts.net/blog/introducing-awooSun, 28 Jul 2019 11:00:00 GMT
title: Animate code with awoo!
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/introducing-awoo
feedurl: https://phaazon.net/blog/feed
content:
<p>Lately, I’ve been wanting to re-write demoscene-like applications. Not in the same mood and way as
I usually did, though. Instead, I want to build small things for people to play with. A bit like
small and easy to use audiovisual experiences (it can be seen as small video games for instance,
but focused on the artistic expression as some games do).</p>
<p>See, the type of application you need to write to make a demo is a bit special. Most programs out
there are often either <em>oneshot</em> or <em>reactive</em>.</p>
<ul>
<li>A CLI program, an encoder/decoder, a math equation solver or a script are often <em>oneshot</em>
programs. You give them a list of inputs and parameters and they do something based on their
inputs. They’re are most of the time non-interactive.</li>
<li>Programs such as GUIs, editors, video games, simulations etc. are often <em>reactive</em>, meaning that
they mostly sleep until an event happen (a key press, mouse movement, etc.). For some, sleeping
is reduced because they have a lot to do (handling network connections, performing I/O
operations, streaming audio, rendering frames to the screen, etc.).</li>
</ul>
<p>That binary split is not absolute, of course. Some programs don’t belong to any of the two prevous
sections (think of a driver, a firmware, etc.). For the purpose of this article, though, it will be
enough.</p>
<p>At some extent, we could state that a demoscene production, a video game or a simulation are also
<em>reactive</em> programs. They mostly react to time as well as user interaction, network events, etc..
About time, you could imagine dividing time passing with very small quanta and picture time as a
virtual clock that <em>ticks</em> such quanta. Every time a new quantum of time is emitted, the whole
simulation is notified and reacts. However, representing that problem this way is not necessarily
the best idea. See, a function of time is often a continuous function and then, it has an output
value for any value of its input that lies in its domain. That property is interesting for us as we
can get a value at any time with an arbitrary precision (the boundary being the precision at which
we can represent a number on a computer).</p>
<p>The thing is, the kind of program we want generates its own inputs based on, mostly, the speed at
which the hardware it’s running on is able to render a complete frame. The faster the more accurate
we sample from that continuous function. That is actually quite logical: more FPS means, literally,
more images to sample. The difference between two images will get less and less noticeable as the
number of FPS rises. That gives you smooth images.</p>
<p>The “challenge” here is to write code to schedule those images. Instead of taking a parameter like
the time on the command-line and rendering the corresponding image, we will generate a stream of
images and will do different things at different times. Especially in demoscene productions, we want
to synchronize what’s on the screen with what’s playing on the audio device.</p>
<h1>The overall idea</h1>
<p>From my point of view, we need at least two mechanisms of synchronization:</p>
<ul>
<li>A <em>high-level</em> synchronization mechanism, to state how globally the application will behave.
I also like to see that kind of mechanism as a <em>discrete space</em> problem. You don’t have values
to interpolate or sample from but only several pure values to “jump” from one to another every
time the simulation time passes a given point in time. This kind of synchronization will state
things like:
<ul>
<li>When the application starts, display this scene and let it play for <em>3 seconds</em>.</li>
<li>Then after <em>3 seconds</em>, switch to this other scene for <em>5 seconds</em>.</li>
<li>After <em>8 seconds</em>, draw this nice little cube and make it crash into a plane for about <em>10
seconds</em>. Repeat that scene once more.</li>
<li>End the application.</li>
</ul>
</li>
<li>A <em>low-level</em> synchronization system. That would be used for a movement, a color change, a
camera change, etc. This kind of synchronization is, to me, a <em>continuous space</em> problem. It
can be solved by sampling from a curve, for instance.</li>
</ul>
<p>Both those problems are solved by two crates I wrote lately. Respectively, <a href="https://crates.io/crates/awoo">awoo</a> and <a href="https://crates.io/crates/splines">splines</a>.
This blog post is about <a href="https://crates.io/crates/awoo">awoo</a>. <a href="https://crates.io/crates/splines">splines</a> already has its own dedicated articles
<a href="https://phaazon.net/blog/splines-introduction">here</a> and
<a href="https://phaazon.net/blog/splines-1.0.0-rc.1">here</a>. Nevertheless, I will make another blog article
about it because I have new ideas I will add to the crate to enrich the <a href="https://crates.io/crates/splines">splines</a> experience.</p>
<h1><a href="https://crates.io/crates/awoo">awoo</a> and the if / else if problem</h1>
<p>Taking on the example of the <em>high-level</em> synchronization described above, one can write quickly the
following naive yet working snippet:</p>
<pre><code class="language-rust">let mut time_ms: f32 = 0.;

loop {
  if time_ms &lt;= 5. {
    // render scene 1
  } else if time_ms &lt;= 8. {
    // render scene 2
  } else if time_ms &lt;= 20. {
    // render scene 1 again
  } else if time_ms &lt;= 25. {
    // render scene 3
  } else
    break; // quit
  }

  time_ms += 0.01; // 100 FPS
}
</code></pre>
<p>That code is typical in demoscene production when we have to rush or even if we have a few scenes
to write. However, it has several problems:</p>
<ul>
<li>It’s not really elegant. All those <code>if</code>s seem like naive and not necessary code.</li>
<li>The more time passes, the more conditions and branching we will do. What it means is that even
if it’s not noticeable (testing floating-point numbers like that is really fast so it won’t
change much), it’s easy to get that the second scene requires two tests in order to be rendered
while scene 3 requires four and scene 1 requires <em>either</em> one or three!</li>
<li>We notice that, every time a condition evaluates to <code>true</code>, all the previous branches can be
completely discarded — we don’t have to test them anymore! — because time will never go
backwards in a simulation (that is a strong assumption and it’s not true if you’re debugging,
but for a release application, it is).</li>
</ul>
<p>So, how can we do better? The idea is actually pretty simple. We want a very simple form a
<a href="https://en.wikipedia.org/wiki/Finite-state_machine">finite-state machine</a>. In our case, the <em>states</em> are just what’s inside our <code>if</code>s; the initial
state is the first scene being rendered and the transitions are a predicate on the current time.
Straightforward, right?</p>
<p>The idea of <a href="https://crates.io/crates/awoo">awoo</a> is exactly that: allowing you to write the previous code like this:</p>
<pre><code class="language-rust">let windows = vec![
  Window::new(0., 5.).map(|_| println!(&quot;hey, it’s scene 1!&quot;)),
  Window::new(5., 8.).map(|_| println!(&quot;hey, it’s scene 2!&quot;)),
  Window::new(8., 20.).map(|_| println!(&quot;hey, it’s scene 1 again!&quot;)),
  Window::new(20., 25.).map(|_| println!(&quot;hey, it’s scene 3!&quot;)),
];
let mut scheduler = SequentialScheduler::new(
  SimpleF32TimeGenerator::new(0., 0.01),
  windows
);

scheduler.schedule();
</code></pre>
<p>The code is now declarative and easier to read. Internally, the <code>SequentialScheduler</code> used here
will make a single test to know which code it has to run. The implementation is not the typical
implementation you would find for a FSM (<a href="https://en.wikipedia.org/wiki/Finite-state_machine">finite-state machine</a>), which uses a graph, but it’s akin.</p>
<p>You might be wondering why we do that <code>map</code> stuff instead of creating a <code>Window</code> directly with the
actions. The answer is simple: a <code>Window</code> doesn’t hold any actions. That allows for creating windows
via JSON, for instance, without having to deal with closures (I have no idea how that would even be
possible with JSON). The idea is then to <em>zip</em> your windows to your actions by using a <em>hashmap</em>,
for instance. This following snippet showcases exactly that
(fully available <a href="https://github.com/phaazon/awoo/blob/master/examples/json-driven.rs">here</a>):</p>
<pre><code class="language-rust">const WINDOWS: &amp;str = r#&quot;
{
  &quot;a&quot;: {
    &quot;start&quot;: 0,
    &quot;end&quot;:   3
  },
  &quot;b&quot;: {
    &quot;start&quot;: 3,
    &quot;end&quot;:  10
  }
}&quot;#;

let windows: HashMap&lt;String, Window&lt;f32&gt;&gt; = from_str(WINDOWS).expect(&quot;cannot deserialize windows&quot;);
let a = windows.get(&quot;a&quot;).unwrap().map(|t| println!(&quot;in a: {}&quot;, t));
let b = windows.get(&quot;b&quot;).unwrap().map(|t| println!(&quot;in b: {}&quot;, t));
}
</code></pre>
<p>What gets interesting is that you can write your own time generator to manipulate the simulation in
other ways — and you can also use different schedulers regarding what you do with time. For
instance, you can imagine implementing a time generator that gets time from a HTTP request, a
keyboard, a network socket, etc. and then control your simulation with external stimuli.</p>
<p>What happens when the escape key is pressed and that you need to stop the simulation in order to
quit? Simple: you need an interruptible scheduler. <a href="https://crates.io/crates/awoo">awoo</a> offers that as well in this form:</p>
<pre><code class="language-rust">use std::sync::mpsc::channel;

let (sx, rx) = channel();
let mut scheduler = create_your_scheduler();

scheduler.interruptible_with(move |_| {
  // here, the closure’s argument is the time at which the scheduler is checking for interruptions
  if let Ok(_) = rx.try_recv() {
    Interrupt::Break
  } else {
    Interrupt::Continue
  }
});

scheduler.schedule();
</code></pre>
<p>Here, we use the spinning loop of the scheduler to check for interruptions in a straight-forward
way.</p>
<p>So far, I have to admit I haven’t digged the <code>async</code>, <code>await</code> and <code>Future</code> concepts in Rust too
much. For a single reason: discussions around those concepts have been heated and I will wait for an
official announcement of the feature. Schedulers, especially as simple as the ones in <a href="https://crates.io/crates/awoo">awoo</a>, don’t
necessarily requires such IO features but the interruptible feature might. To me, the current
implementation of interruptible schedulers in <a href="https://crates.io/crates/awoo">awoo</a> is sufficient, especially for animation
purposes — I might even add that feature directly in <a href="https://crates.io/crates/awoo">awoo</a> so that you don’t have to do it by hand.</p>
<h1>About the scope of the crate</h1>
<p>Currently, the crate’s scope is very narrow — and I actually like that. A tight and small scope
implies a better visibility about what the crate must do and how it must do it. The crate is
currently simple and it might get more and more complex stuff as needs appear. As I always tell
other developers and engineers, I don’t like to overthink too much features I don’t even need.
Obviously, it’s important to keep planning possible future additions… But not too much. This is why
that crate’s scope, if augmented, will only and always revolve around the concept of scheduling
animation code. It’s currently an experimental crate and I’m trying to write demos with it, so we’ll
see what time thinks about it.</p>
<p>So that’s all for me for today. I hope you liked it. Keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/software_meta_designSun, 13 Sep 2015 00:00:00 GMT
title: Thoughts about software meta-design
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/software_meta_design
feedurl: https://phaazon.net/blog/feed
content:
<p>I’ve been thinking of writing such an article for a while. A few weeks ago, I got contacted by
people who wanted to know more about my experience with
<a href="https://github.com/phaazon/luminance">luminance</a> so that they can have more hindsight about their
own APIs and products.</p>
<p>I came to the realization that I could write a blog entry to discuss designs decisions and, at some
extent, what a good design entails. Keep in mind it’s only personal thoughts and that I won’t talk
for someone else.</p>
<h1>Elegancy</h1>
<p>I love mathematics because they’re elegant. Elegancy implies several traits, among <em>simplicity</em>,
<em>flexibility</em> and <em>transparency</em>. They solve problems with very nice abstractions. In
mathematics, we have a concept that is – astonishingly – not very spread and barely known outside
of math geeks circles: <em>free</em> objects.</p>
<p>The concept of <em>free</em> is a bit overwhelming at first, because people are used to put labels and
examples on everything. For instance, if I say that an object is <em>free</em>, you might already have
associated some kind of <em>lock</em> to that object, so that you can get why it’s <em>free</em>. But we’re
mistaken. We don’t need <em>locks</em> to define what <em>free</em> implies. In mathematic, a <em>free</em> object is an
object that can’t be defined in terms of others. It’s a bit like a <em>core</em> object. It’s <em>free</em>
because it can be there, no matter what other objects are around. It has no dependency, it doesn’t
require no other interaction. You can also say that such an object is <em>free</em> of extra features that
wouldn’t be linked to its nature.</p>
<p>This <em>free</em> property is a very interesting property in mathematics, because it’s surprisingly
simple! We can leverage that mathematic abstraction to software design. I like keeping my softwares
as much <em>free</em> as possible. That is – with a more human language to say it – constraining them to
keep low responsibilities about what they’re doing.</p>
<h1>Responsibility domains</h1>
<p>The important thing to keep in mind is that you should, at first, define what the responsibility
domain is all about. Let’s say you’d like to create a library to implement audio effects, like the
<a href="https://en.wikipedia.org/wiki/Doppler_effect">Doppler effect</a> – that effect actually exists for
any kind of wave, but it’s interesting to synthetize it for a sound-related application. If you end
up writing functions or routines to play sound or to load audio samples, you’re already doing it
wrong! You’d have violated your reponsibility domain, which is, <em>“audio effects”</em>. Unfortunately,
<strong>a lot</strong> of libraries do that. Adding extra stuff – and sometimes, worse; relying on them!</p>
<p>A lot of people tend to disagree with that – or they just <em>ignore</em> / <em>don’t know</em>. There’re plenty
of examples of libraries and softwares that can do everything and nothing. For instance, take
<a href="http://www.qt.io/">Qt</a> – pronounce <em>cute</em> or <em>cutie</em>. At first, <em>Qt</em> is a library and an API to
build up <em>GUIs</em> – Graphical User Interfaces – and handle windows, events and so on. Let’s have a
look at the documentation of modules, <a href="http://doc.qt.io/qt-5/qtmodules.html">here</a>.</p>
<p>You can see how the responsibility domain is <strong>huge</strong>! GUI, radio, audio, video, camera, network,
database, printing, concurrency and multithreading… <em>Qt</em> isn’t a library anymore; it’s a whole new
language!</p>
<p>People tend to like that. <em>“Yeah, I just have to use Qt, and I can do everything!”</em>. Well, that’s a
point. But you can also think it another way. Qt is a very massive “library” you’ll spend hours
reading the documentation and will use a lot of different classes / functions from different
aspects. That doesn’t compose at all. What happens when you want to – or when you don’t have the
choice? – use something else? For instance, if you want to use a smaller–but–dedicated threading
library? What happens if you want to use a database service you wrote or that you know it’s great?
Do you wipeout your Qt use? Do you… try to make both work in harmony? If so, do you have to write a
lot of boilerplate code? Do you forget about those technologies and fallback on Qt? Do the concepts
map to each others?</p>
<p>The problem with massive libraries is the tight bound it creates between the libraries and the
developers. It’s very hard with such libraries to say that you can use it whenever you want because
you perfectly know them. You could even just need a few things from it; like, the <em>SQL</em> part. You’ll
then have to install a lot of code you’ll perhaps use 10% of.</p>
<h1>KISS</h1>
<p>I love how the <em>free</em> objects from mathematics can be leveraged to build simpler libraries here. The
good part about <em>free</em> objects is the fact that they don’t have any extra features embedded. That’s
very cool, because thanks to that, you can reason in terms of such objects <em>as-is</em>. For instance,
<a href="http://www.openal.org">OpenAL</a> is a very <em>free</em> audio library. Its responsibility domain is to be
able to play sound and apply simple effects on them – raw and primary effects. You won’t find
anything to load music from files nor samples. And that’s very nice, because the API is <strong>small</strong>,
<strong>simple</strong> and <strong>straight-forward</strong>.</p>
<p>Those adjectives are the base of the <a href="https://en.wikipedia.org/wiki/KISS_principle">KISS principle</a>.
The ideas behind <em>KISS</em> are simple: keep it simple and stupid. Keep it simple, because the simpler
the better. A too complex architecture is bloated and ends up unmaintainable. Simplicity implies
elegancy and then, flexibility and composability.</p>
<p>That’s why I think a good architecture is a small – in terms of responsibility – and simple one. If
you need complexity, that’s because your responsibility domain is already a bit more complex than
the common ones. And even though the design is complex for someone outside of the domain, for the
domain itself, it should stay simple and as most straight-forward as possible.</p>
<h1>API</h1>
<p>I think a good API design is to pick a domain, and stick to it. Whatever extra features you won’t
provide, you’ll be able to create other libraries to add those features. Because those features will
also be <em>free</em>, they will be useful in other projects that you don’t even have any idea they exist!
That’s a very cool aspect of <em>free</em> objects!</p>
<p>There’s also a case in which you have to make sacrifices – and crucial choices. For instance,
event-driven programming can be implemented via several techniques. A popular one in the functional
programming world nowadays is <a href="https://wiki.haskell.org/Functional_Reactive_Programming">FRP</a>. Such
a library is an <em>architectural codebase</em>. If you end up adding <em>FRP</em>-related code lines in your
networking-oriented library, you might be doing it wrong. Because, eh, what if I just want to use
imperative event-driven idioms, like <a href="https://en.wikipedia.org/wiki/Observer_pattern">observers</a>?
You shouldn’t integrate such architectural design choices in specific libraries. Keep them <em>free</em>,
so that everyone can quickly learn them, and enjoy them!</p>
<p>I like to see good-designed libraries as a set of very powerful, tiny tools I can compose and move
around freely. If a tool gets broken or if it has wrong performances, I can switch to a new one or
write my very own. Achieving such a flexibility without following the <em>KISS principle</em> is harder or
may be impossible to reach.</p>
<p>So, in my opinion, we should keep things simple and stupid. They’re simpler to reason about, they
compose and scale greatly and they of course are easier to maintain. Compose them with architectural
or whatever designs in the actual final executable project. Don’t make premature important choices!</p>


guid: https://strongly-typed-thoughts.net/blog/abstracting_over_shaderFri, 14 Nov 2014 00:00:00 GMT
title: Abstracting shader – Haskell ash library
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/abstracting_over_shader
feedurl: https://phaazon.net/blog/feed
content:
<h1>Foreword</h1>
<h2>Abstracting what?</h2>
<p><a href="http://en.wikipedia.org/wiki/Shader">Shaders</a> are very common units in the world of graphics. Even though we’re used to using them for shading[^shading] purposes, they’re not limited to that. Vulgarisation has ripped off the meaning up and down so much that nowadays, a <strong>shader</strong> might have nothing related to shading. If you’re already doing some graphics, you may know <em>OpenGL</em> and its <em>compute shaders</em>. They have nothing to do with shading.</p>
<p>You might also already know <a href="https://www.shadertoy.com/">shader toy</a>. That’s a great place to host cool and fancy <em>OpenGL</em> shaders[^shadertoy_shaders]. You write your shaders in GLSL[^GLSL] then a GLSL compiler is invoked, and your shader is running on the GPU.</p>
<h2>The problem with source based shaders</h2>
<p>So you write your shader as a source code in a host language, for instance in <em>C/C++</em>, <em>Java</em>, <em>Haskell</em>, <em>whatever</em>, and you end up with a shader running on GPU.</p>
<p>There’re two nasty issues with that way of doing though:</p>
<ul>
<li>the shader is compiled at runtime, so if it contains error, you’ll know that after your application starts ;</li>
<li>you have to learn a new language for each target shader compilers.</li>
</ul>
<p>They’re both serious issues I’m going to explain further.</p>
<h3>Issue n°1: compiled at runtime</h3>
<p>This is problematic for a good reason: a lot of shaders are application dependent. <strong>Shadertoy</strong> is a nice exception, just like modeling tools or material editors, but seriously, in most applications, end users are not asked the shaders to run with. In a game for instance, you write all the shaders while writing the game, and then release the whole package.</p>
<blockquote>
<p><em>Yeah… What’s about additional content? Per-map shaders, or that kind of stuff?</em></p>
</blockquote>
<p>Those shaders are like resources. That doesn’t imply using them as is though. We could use dynamic relocatable objects (<strong>.so</strong> or <strong>.dll</strong>) for instance.</p>
<blockquote>
<p><em>What compile-time compilation gives you?</em></p>
</blockquote>
<p>It gives you something hyper cool: <strong>host language features</strong>. If you have a strongly-typed language, you’ll benefit from that. And that’s a huge benefit you can’t get away from. If you’re writing an incorrectly typed shader, your application / library won’t compile, so that the application won’t react in weird way at run-time. That’s pretty badass.</p>
<h3>Issue n°2: languages, languages…</h3>
<p>This issue is not as important as the first one, but still. If you’re working on a project and you target several platforms (among ones using <em>OpenGL</em>, <em>OpenGL ES</em>, <em>DirectX</em> and a <em>soft renderer</em>), you’ll have to learn several shading languages as well (<strong>GLSL</strong>, <strong>HLSL</strong>[^HLSL]).</p>
<p>In order to solve that, there’re two ways to go:</p>
<ul>
<li>a DSL[^DSL] ;</li>
<li>an EDSL[^EDSL].</li>
</ul>
<p>A DSL is appealing. You have a standalone language for writing shaders, and backends for a compiler/language. However, that sounds a bit overwhelming for such an aim.</p>
<p>An EDSL is pretty cool as well. Take a host language (we’ll be using <strong>Haskell</strong>) and provide structure and construction idioms borrowed from such a language to create a small embedded one. That is the solution I’m going to introduce.</p>
<h1>Ash</h1>
<p><strong>Ash</strong> stands for <strong>Abstract Shader</strong>. It’s a Haskell package I’ve been working on for a few weeks now. The main idea is:</p>
<ul>
<li>to provide a typesafe shading language compiled at compile-time;</li>
<li>to provide backends;</li>
<li>to provide a nice and friendly <em>haskellish</em> interface.</li>
</ul>
<p>I guessed it’d be a good idea to share my thoughts about the whole concept, since I reckon several people will be interested in such a topic. However, keep in mind that Ash is still a big work in progress. I’m gonna use several blog entries to write down my thoughts, share it with you, possibly enhance Ash, and finally release a decent and powerful library.</p>
<p>If you’re curious, you can find Ash <a href="https://github.com/phaazon/ash">here</a>.</p>
<h2>Basics</h2>
<p><strong>Ash</strong> is a library that provides useful tools to build up <em>shaders</em> in <strong>Haskell</strong>. In <strong>Ash</strong>, a <em>shader</em> is commonly function. For instance, a vertex shader is a function that <em>folds</em> vertex components down to other ones – possibly <em>maps</em>, but it could <em>add</em>/<em>remove</em> components as well – and <em>yields</em> extra values for the next stages to work with.</p>
<p>You write a shader with the Ash EDSL then you pass it along to a <strong>backend compiler</strong>.</p>
<p>Here are two examples. In order for you to understand how Ash works, I’ll first write the GLSL (330 core) shader, then the Ash one.</p>
<h3>First example: a simple vertex shader</h3>
<p>Let’s write a vertex shader that takes a position and a color, and projects the vertex using a perspective matrix, a view matrix and the object matrix of the object currently being rendered and passes the color to the next stage:</p>
<pre><code>#version 330 core

in vec3 pos;
in vec4 col;

out vec4 vcol;

uniform mat4 projViewModel;

void main() {
  vcol = col; 
  gl_Position = projViewModel * vec4(pos, 1.);
}
</code></pre>
<p>And now, the Ash one:</p>
<pre><code>vertexShader :: Ash (M44 Float -&gt; V3 Float :. V4 Float -&gt; V4 Float :. V4 Float)
vertexShader = lam $ \proj -&gt; lam $ \v -&gt;
  let pos :. col = v
  in proj #* v3v4 pos 1 :. col
</code></pre>
<p><code>Ash</code> is the type used to lift the shading expression up to Haskell. You use it to use the EDSL. It actually represents some kind of HOAST[^HOAST].</p>
<p>Then, you can find <code>M44</code>, <code>V3</code>, <code>V4</code> and <code>(:.)</code>.</p>
<p><code>M44</code> is the type of <strong>4x4 matrices</strong>. Since projection matrix, view matrix and model matrix are all 4x4 floating matrix, <code>M44 Float</code> makes sense.</p>
<p><code>V3</code> and <code>V4</code> represents 3D and 4D vectors, respectively. <code>V3 Int</code> is three ints packed in a vector as well as <code>V4 Float</code> is four floats packed in a vector. You’ll also meet <code>V2</code>, which is… the 2D version.</p>
<p><code>(:.)</code> is a <a href="https://www.haskell.org/ghc/docs/7.2.1/html/users_guide/data-type-extensions.html#infix-tycons">type operator</a> used to build tuples. You can see <code>(:.)</code> as a generalized <code>(,)</code> – the default Haskell pair type – but <code>(:.)</code> is more power full since it can flatten expressions:</p>
<pre><code>a :. (b :. c) = a :. b :. c
</code></pre>
<p>The <code>(:.)</code> has a lot of uses in Ash. In our cases, a chain of <code>(:.)</code> represents a vertex’ components.</p>
<p>So our <code>vertexShader</code> value is just a function that takes a matrix and a vertex (two components) and outputs two values: the new position of the shader, and the color. Let’s see the body of the function.</p>
<pre><code>lam $ \proj -&gt; lam $ \v -&gt;
</code></pre>
<p>This is a pretty weird expression, but I haven’t found – yet? – a better way to go. <code>lam</code> is a combinator used to introduce lambdas in the EDSL. This expression then introduces a lambda that takes two values: <code>proj</code> and <code>v</code>. You can read that as:</p>
<pre><code>\proj v -&gt;
</code></pre>
<p>Next:</p>
<pre><code>let pos :. col = v
</code></pre>
<p>This is the tricky part. That <em>let expression</em> extracts the components out of the vertex and binds them to <code>pos</code> and <code>col</code> for later use.</p>
<pre><code>in proj #* v3v4 pos 1 :. col
</code></pre>
<p><code>(#*)</code> is a cool operator used to multiply a matrix by a vector, yielding a new vector.</p>
<p><code>(v3v4)</code> is a shortcut used to to build a <code>V4</code> using a <code>V3</code> by providing the missing value – here, <code>1</code>. You’ll find similar functions, like <code>v2v3</code> and <code>v2v4</code>, to respectively build a <code>V3</code> from a <code>V2</code> by providing the missing value and build a <code>V4</code> from a <code>V2</code> by providing the two missing values.</p>
<p>We finally wrap the result in a tuple <code>(:.)</code>, and we’re done.</p>
<h2>Features</h2>
<p>Ash embeds regular linear expressions (vectors, matrix), textures manipulation, tuples creation, let-bindings, lambda functions (they represent shader stages up to now), and a lot of other features.</p>
<p>Each feature is supposed to have an implementation in a given <em>backend</em>. For instance, in the GLSL backend, a lambda function is often turned into the well done <code>main</code> function. Its parameters are expanded to as <code>in</code> values, and
control parameters are <code>uniform</code> variables.</p>
<p>Each backend is supposed to export a <code>compile</code> function – the name may varies though. However, each backend is free to compiles to whatever smart they think is. For instance, compiling an Ash shader to GLuint (shader stage) is not very smart since it would use <code>IO</code> and handles error a specific way we don’t want it to do. So the GLSL compiler is a function like <code>glslCompile :: Ash … -&gt; Either CompilationError String</code>, and the <code>String</code> can be used as a regular GLSL source code string you’ll pass to whatever implementation of shader you’ve written.</p>
<h1>What’s next?</h1>
<p>I need to finish the implentation of the EDSL, and write the whole GLSL 330 compiler. If it’s a success, I’ll accept pull-requests for other famous compilers (other GLSL version compilers, HLSL, and so on and so forth).</p>
<p>Once that done, I’ll write a few other blog entries with example as a proof-of-concept :)</p>
<p>[^shading]: Shading is the process in which <em>primitives</em> (sets of <em>vertices</em>) are turned into colors (i.e <em>fragments</em>, a.k.a. <em>pixels</em> or <em>texels</em>).</p>
<p>[^shadertoy_shaders]: Actually, they’re <strong>fragment shaders</strong>.</p>
<p>[^GLSL]: <a href="https://www.opengl.org/documentation/glsl/">OpenGL Shading Language</a>.</p>
<p>[^HLSL]: <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/bb509638(v=vs.85).aspx">High Level Shading Language</a>.</p>
<p>[^DSL]: <a href="http://en.wikipedia.org/wiki/Domain-specific_language">Domain-Specific Language</a>.</p>
<p>[^EDSL]: Embedded Specific Language.</p>
<p>[^HOAST]: <a href="http://en.wikipedia.org/wiki/Higher-order_abstract_syntax">High-Order Abstract Syntax tree</a>; for the purpose of this paper, you don’t have to fully understand them to get your feet wet with Ash (which is cool, right? :) ).</p>


guid: https://strongly-typed-thoughts.net/blog/development-environmentsSun, 26 Jun 2022 15:01:00 GMT
title: Development Environments
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/development-environments
feedurl: https://phaazon.net/blog/feed
content:
<p>It’s 2022. Today, developing a library, an application or a service is a totally different experience from what it used
to be only a few years ago. Every professional engineer or programmer is expected to know tools like <code>git</code>, a coding
editor, a terminal to run various commands, like compiling, testing, etc. You might also know a lot of people (if you
are not one already) using modern editors and IDEs, such as <strong>VS Code</strong>, <strong>IntelliJ</strong>, etc.</p>
<p>I belong to the category of people who enjoy <em>composing</em> tools. That is, I don’t use an IDE: I make my own IDE by
combining several tools I like. In the end, we all use “IDEs” (we all need some kind of <em>development environment</em>).
For the sake of generality, I will call those environments “DE” for <em>development environments</em>.</p>
<p>The tools I use are, basically:</p>
<ul>
<li>A terminal. Lately, I’ve been enjoying <a href="https://sw.kovidgoyal.net/kitty">kitty</a> a lot, for it allows me to do <em>almost</em> everything I do with <a href="https://github.com/tmux/tmux">tmux</a>
(read below).</li>
<li>A shell. I’m using <a href="https://www.zsh.org">zsh</a> mostly but I’ve been playing with others, such as <a href="https://www.nushell.sh">nushell</a>. I’m dissatisfied with <a href="https://www.zsh.org">zsh</a>
because I think it’s way more too complicated (I barely use 10% of its features).</li>
<li>A text editor. I’ve been using lots lately, but mainly, <a href="https://www.vim.org">vim</a> / <a href="https://neovim.io">neovim</a>.</li>
<li>A git client. I simply use the <a href="https://git-scm.com">git</a> CLI (<em>Command Line Interface</em>) program, but also <a href="https://github.com/tpope/vim-fugitive">fugitive</a> in <a href="https://neovim.io">neovim</a>.</li>
<li>A couple of other programs, such as <a href="https://github.com/junegunn/fzf">fzf</a>, CLI of various package managers and compilers, etc.</li>
<li>I have plugins in <a href="https://neovim.io">neovim</a> for note taking and task management, but I’ve also been using <a href="https://www.notion.so">Notion</a> at work lately to
try it out.</li>
<li>I used to use <a href="https://github.com/tmux/tmux">tmux</a> a lot but I’m moving away from it, so I’m basically left with <a href="https://sw.kovidgoyal.net/kitty">kitty</a>’s own way to do the same
thing.</li>
</ul>
<p>The more that I think about all the things I use and all the things people enjoying modern environments use, the more I
feel I try to “force” a workflow that used to be the right one a couple of years ago, but not necessarily today. I’m the
kind of person who <em>constantly</em> put things into perspective and try to balance my decisions regarding the time I have,
the budget (for work) and the knowledge. A couple of months / years ago, I spent a lot of time on <a href="https://www.gnu.org/software/emacs">emacs</a> and realized
that I needed to review my perspective, because even though <a href="https://www.gnu.org/software/emacs">emacs</a> is super old, it’s fascinating how bleeding edge it
still feels (and I can’t even imagine how bleeding edge it was twenty years ago!).</p>
<p>I want to use this blog article to ask questions and not necessarily answer them, but discuss about them. The questions
are about “how we should be developing in the modern era” and is not about “is <a href="https://www.gnu.org/software/emacs">emacs</a> better than <a href="https://neovim.io">neovim</a>” or that
kind of useless debates.</p>
<!-- vim-markdown-toc GFM -->
<ul>
<li><a href="#is-tooling-composition-that-any-good">Is tooling composition that any good?</a>
<ul>
<li><a href="#the-frozen-world">The frozen world</a></li>
<li><a href="#the-nightmare-that-is-composition-iteration">The nightmare that is composition iteration</a></li>
<li><a href="#a-small-word-on-tuis-the-land-of-the-unicode-hacks">A small word on TUIs, the land of the unicode hacks</a></li>
<li><a href="#summary-of-the-problems">Summary of the problems</a></li>
</ul>
</li>
<li><a href="#what-would-be-the-next-perfect-productivity-platform-in-2022">What would be the (next) perfect productivity platform in 2022?</a>
<ul>
<li><a href="#the-terminal-needs-to-change">The terminal needs to change</a></li>
<li><a href="#editing-needs-to-come-back-to-editing">Editing needs to come back to editing</a></li>
<li><a href="#applications-need-to-compose-in-a-new-modern-way">Applications need to compose in a new, modern way</a></li>
<li><a href="#applications-need-to-change">Applications need to change</a></li>
</ul>
</li>
</ul>
<!-- vim-markdown-toc -->
<h1>Is tooling composition that any good?</h1>
<p>The first question I’m wondering is that whether using several tools and composing them is actually good in 2022. For
instance, the best example I have in mind is <a href="https://kubernetes.io">Kubernetes</a> and <a href="https://github.com/junegunn/fzf">fzf</a>. At work, I have a workflow where I use <a href="https://kubernetes.io">Kubernetes</a>
and filter/select its output via <a href="https://github.com/junegunn/fzf">fzf</a> to run other commands. I do the same with <a href="https://git-scm.com">git</a>, btw. I have some scripts where I
can do something like:</p>
<pre><code class="language-sh">gr -s # equivalent to git branch | fzf | git rebase origin --auto-stash
gs # equivalent to git branch | fzf | git switch
gm # equivalent to git branch | fzf | git merge
kubectx # switch via FZF k8s clusters (kubectl command)
kubens # same but with k8s namespaces
# etc. etc.
</code></pre>
<h2>The frozen world</h2>
<p>All of that seems to be pretty good. We can leverage the power of small tools such as <code>git</code>, <code>kubectl</code> and <code>fzf</code> and
compose them. Shell pipes allow to connect applications together, reading and writing from and to <code>stdin</code> and <code>stdout</code>.</p>
<p>The “main platform”, the “DE” is then the terminal and the shell, and building a DE requires two main ingredients:</p>
<ul>
<li>A set of programs, each of them doing <em>one thing right</em>. <code>fzf</code> is excellent at building a list of items and let the
user pick one of them, for instance. <code>kubectl</code> allows to operate Kubernetes resources one operation by one operation.</li>
<li>Some “glue” to combine the results of the programs. This is typically the shell, with its pipes and output pushed to
the terminal.</li>
</ul>
<p>However, I think that even though all of that seems pretty great, it suffers from a major drawback: everything is text
and everything is static. See, all of the programs I mentioned here are CLI, not TUI (<em>Text User Interface</em>). That is,
the UX is the following:</p>
<ol>
<li>The user enters a command in their terminal, starting with the program name, passing arguments.</li>
<li>The program runs, might have side effects on remote resources, etc.</li>
<li>The program finishes and displays its results.</li>
</ol>
<p>That way of working has been a default in the industry for decades. If the program printed more than one terminal page,
you will have to scroll to find your information back up. Once you find the information you need, how do you pick that
information? If you are using a terminal that doesn’t support hinting, you will have to use your mouse (which completely
defeats the purpose of using CLI / TUI applications to me) or the keyboard-base navigation system of your terminal to
make selections, which is, let’s be honest, really poor in most terminal. If, like me, you use a terminal supporting
hinting (i.e. being able to enter a mode that will allow you to visually select parts of the screen by simply typing a
few characters — similar to my <a href="https://github.com/phaazon/hop.nvim">hop.nvim</a> plugin, but for the terminal), you will be able to copy parts of the screen,
but that’s all. If a program returned a JSON array, it will cost you a lot of keystrokes to reduce the list and, for
instance, use an item in the array as input of another <code>curl</code> request, for instance.</p>
<p>The <a href="https://www.nushell.sh">nushell</a> has a very interesting concept to solve that kind of problem. Programs write lines of text as output, but
they can also write <em>cells</em>, which is a <a href="https://www.nushell.sh">nushell</a> encoding that allows it to “speak the same language.” It comes with
some built-in programs such as <code>open</code> that will be able to recognize a lot of formats (JSON, XML, YAML, TOML, etc.) and
convert them into <em>cells</em>, allowing to display them in the terminal in a very convenient and “smart” way. However, it
doesn’t change the next problem: composition iteration.</p>
<h2>The nightmare that is composition iteration</h2>
<p>I think pretty much every programmer out there can relate to the following: you will oftentimes run a command, see its
result, then run the command again, piping it to a filter command or sort or field extractor like [jq]. If the output is
too long, it’s likely you will output into a file and will work on that file (which is basically a cached version of
your command output ;) ).</p>
<p>Iterating on a “session” is a pretty bad experience in a terminal to me. To my knowledge, it would be quite easy to fix
by simply allowing to automatically cache the result of all commands, and get access to it via a shell variable (such as
what we have with <code>$?</code> for the last command status, for instance). Some hacks exist to <strong>recompute</strong> the last command
(i.e. <code>$(!!)</code>) but it will still re-run the last command.</p>
<p>I thought <a href="https://www.nushell.sh">nushell</a> supported that, because the variable support and Nu language are pretty excellent, but to my
surprise, that shell doesn’t have that. Maybe it’s a technical challenge, but I really doubt it. We work with computers
with at least 8 GB of RAM these days and most of us have between 16 and 32 GB of RAM. I don’t think it would be that
hard to support.</p>
<p>But even if we had that caching… it would still require us to run a command, look at a static text, run a filter command
on it, look at the new output that would be appended to the screen… it doesn’t seem super nice in terms of UX to me. And
imagine having to change the second or third command in the pipe list while you already have six pipes. Duh. The
problem, to me, is that the terminal application scope is being overloaded. The CLI is great when we want to run a
command and expect a <em>report</em>. As soon as we need to work with <em>data</em>, I think the CLI is already lacking features.</p>
<h2>A small word on TUIs, the land of the unicode hacks</h2>
<p>And now, let’s talk about TUIs. As a lot of people, I’m using TUIs, so don’t get what I’m about to say twisted. I use
<a href="https://neovim.io">neovim</a> as my daily editor driver, and so far, I haven’t really found a way out and use something else. However, it’s
not because nothing better exists that what we use is perfect (far from that).</p>
<p>I have always despised TUIs. I think the idea, which dates from a while, was a fun and interesting idea back then, when
graphical elements and graphics kits were not mature and not a thing yet. But today, we know we can have graphical
elements that align at the pixel level. We know that we can have real vector images in applications. We can use the GPU
to draw arbitrary shapes and some terminals (text-based only!) use the GPU to render glyphs (text), and then people use
TUIs that will use the glyphs to mimic graphical elements! That hitches me!</p>
<p>I dislike TUIs because they are sub-optimal versions of GUIs. And here, I think a lot of people make a confusion with
UX and UI. A GUI can be beautiful, rich of graphical elements, and have a terrible UX (it’s the case most of the time).
People writing GUIs tend to focus on mouse-driven interactions… which doesn’t have to be. So people tend to associate
GUIs with mouse-only workflows, while a GUI can completely do everything a TUI does… but it can <em>also</em> do much better.</p>
<p>A couple of arguments against TUIs:</p>
<ul>
<li>Graphical elements are faked by using unicode symbols (drawing blocks), which implies having all the elements laid out
on a display cell grid. Oftentimes, the unicode symbols will not be correctly aligned, resulting in graphical
glitches, or simply ugly pop-ups borders, etc.</li>
<li>Also, still about unicode glyphs, if the application doesn’t implement it correctly, that text is just “part of the
text of the application”, so copy-pasting can be challenging and very boring.</li>
<li>Since everything has to be put on a display cell grid, you have no way to have pixel-based movements, alignment or
anything based on pixel. The smallest unit, the basic primitive, is a unicode character. About that, I think the most
interesting “fake” thing is <a href="https://github.com/wfxr/minimap.vim">wfxr/minimap.vim</a>, which is a minimap for <a href="https://neovim.io">neovim</a>
in a terminal. And you can’t do any better than that, because, well, you can’t go any smaller / different than a
glyph.</li>
<li>Applications run in a terminal, which drives the performance of the TUI application. That results in non-deterministic
experiences. Basically, you can write a super great application that will run like an elder snail on some terminal
emulators.</li>
</ul>
<h2>Summary of the problems</h2>
<p>So let’s do a quick recap of what’s wrong with today terminals:</p>
<ul>
<li>TUIs are not rich enough in terms of graphical experience, and everything graphical (splits, lines, pop-ups, etc.) are
somehow faked.</li>
<li>CLI-based composition often ends up being a nightmare of running the same command over and over by adding more piped
commands.</li>
<li>A terminal is a static world, where (besides TUIs) everything is just static text and nothing changes unless you run a
command again, wiping the previous result and replacing it with the new one.</li>
</ul>
<p>Those are the three main pain points I feel about using a terminal. Now, you might already have used something like a
monitor application (I work at <a href="https://www.datadoghq.com">DataDog</a>, so I’m used to them) or used a web browser or native GUI and see that you can
have animations, drop-down lists that actually look good, etc. Putting all that in contrast with terminals makes me a
bit sad, especially in 2022.</p>
<h1>What would be the (next) perfect productivity platform in 2022?</h1>
<p>My latest blog articles are centered around that topic:</p>
<ul>
<li><a href="https://phaazon.net/blog/editors-in-2020">My thoughts about editors in 2020</a></li>
<li><a href="https://phaazon.net/blog/editors-in-2021">My thoughts about editors in (early) 2021</a></li>
</ul>
<p>So it’s been an important topic to me lately. I have been thinking about productivity platforms for a while now, and the
conclusion I came to is as follows.</p>
<h2>The terminal needs to change</h2>
<p>Instead of switching to an IDE like <strong>IntelliJ</strong> or <strong>VS Code</strong>, I still think we need composition, but not in the form
of pipe shells. I think the CLI has to evolve. I have started a project on my spare-time to try and change how programs
would work with their environment. See, a shell is basically just a way to call the <code>exec</code> syscall (<code>execve</code> on Linux
for instance) and set its arguments, environment and standard input / output (plus all the candies you can get from a
shell, such as scripting languages, etc.).</p>
<p>The interpretation of the inputs and outputs of a program is actually completely left to… the running process, which is
most of the time the shell for CLI applications. This is why <a href="https://www.nushell.sh">nushell</a> is such an interesting new shell: it can
interpret <em>more</em> than just lines of text. And I think we need to move in a direction similar to this.</p>
<p>I think we should see more alternative forms of “terminals”. We could even imagine a program that doesn’t have the
concept of “shell” nor “terminal” anymore. It’s easy to imagine having an application that can simply call <code>execve</code> and
do whatever it wants the environment, inputs and outputs. It could have some primitives implemented. For instance,
imagine that a program could be run in your “new terminal” and output its results in a tree view, while other programs
are also running, displaying their own things. You could have a program run without exiting, updating the content of the
tree view or even accepting filters, sorts, field extractor or whatever see fit.</p>
<h2>Editing needs to come back to editing</h2>
<p>Today, text editors are overloaded. Heck, even small editors such as <a href="https://neovim.io">neovim</a> now have a community developing plugins
that have nothing to do with editing. For instance, fuzzy searching files or Git management. When you think about it, if
the platform (i.e. terminal) had better primitives, it should be possible to compose programs in a way that doesn’t
require people to rewrite the same kind of logic (i.e. fuzzy finders) in all the programs they use. <a href="https://github.com/junegunn/fzf">fzf</a> is a good
example of that, but it’s limited to the CLI (you can still use it in TUI but to me it’s a bit hacky). Terminals should
have a primitive to, for instance, run a program in overlay and pass the output to another, running program. That would
allow people to write “pure generic fuzzy finders” once and for all, and let people use the editors they like.</p>
<p>Today, all that logic doesn’t really exist outside of individual and scoped efforts. Yes, you can probably find programs
supporting that, but they will certainly not run in a terminal and will have their own protocols.</p>
<h2>Applications need to compose in a new, modern way</h2>
<p>Composing applications with pipes is easy to wrap your fingers around, but it gets limited and limiting as soon as
you use something like a GUI. Imagine that programs could speak a common language, based on, for instance, local
sockets, to exchange messages in a way that whatever the kind of programs you write, you can still compose them.
Composing two GUIs between each other should be something that is widely supported, and standardized.</p>
<p>Something else with piping applications: it’s limited to <em>how we use CLI applications</em>, which is via the command line.
Running programs in the command line takes the form of:</p>
<pre><code class="language-sh">program &lt;arg1&gt; &lt;arg2&gt; … | another_program &lt;arg1&gt; &lt;arg2&gt;
</code></pre>
<p>This is highly summarized, but you get the idea. CLI programs don’t provide a great discoverability experience, and it’s
harder to iterate on them. For instance, in a GUI, if you filter a list with a predicate, it’s super easy to
“experiment around” by changing the filter name. In a CLI session with piped programs, you are likely required to use
your up arrow on your keyboard (or whatever shortcut) to invoke the last command again, and change the arguments in the
middle of the line. While doing this, you will still have to run all intermediary commands, which are going to perform
side-effects, etc. etc. And if you want to be smarter and use the <code>&gt;</code> or <code>&gt;&gt;</code> operator to cache the results, then you
will still have to remember the name of the file you used, and <code>cat</code> or <code>tee</code> that file again. It gets time to get used
to using that, and I truly think that it’s not really worth it, because today I’m 30 and I’ve been developing since I’m
11, and I still don’t really enjoy that kind of process.</p>
<h2>Applications need to change</h2>
<p>The essence of an application, today, is actually platform-dependent. Heck, it’s even environment dependent. At work,
an “application” can be as simple as a small Python script or be a <a href="https://kubernetes.io">Kubernetes</a> cluster composed of several deployments,
stateful sets and DNS rules. However, when we think about applications we use on a daily basis (what people would call
“heavy” or “native” applications), they are pretty similar to the following:</p>
<ul>
<li>For CLI applications, they are akin to effectful computations. They take inputs (environment, <code>stdin</code>, arguments),
they output something (<code>stdout</code>) and they can have side-effects (calling another program, changing shared memory,
doing stuff on your file system, various kinds of I/O, etc.).</li>
<li>For TUI applications, it’s similar but they don’t read from <code>stdin</code> nor output to <code>stdout</code> in a one-shot way. Instead,
it’s like an <em>event loop</em> that reads from <code>stdin</code> and a <em>render loop</em> that writes to <code>stdout</code>.</li>
<li>For GUI, it depends on the platform but most of the time, GUIs will be closer to the hardware, because they are not
restricted by the protocol they are written for (i.e. CLIs / TUIs cannot really go any lower <code>stdin</code> and <code>stdout</code> in
terms of UX; GUIs can). A GUI application can decide to use <code>OpenGL</code>, <code>Vulkan</code> or something higher such as <code>Gtk</code>,
<code>Qt</code>, or a Web kit.</li>
</ul>
<p>I think that we should change that. Remember that running an application is just a matter of calling <code>execve</code> (or
similar on non-Linux), which basically requires a path to the application, the list of arguments (the <code>argv</code>) and the
environment (<code>envp</code>). Setting the environment is typically done by the shell (and some variables come from the terminal
too), but it could be done completely differently. On the same idea, the <code>argv</code> is filled by the CLI arguments you
provide, but they could be computed and filled in a completely different way. And finally, the interpretation of
<code>stdout</code> (mostly done by the terminal) could be interpreted in a completely different way as well.</p>
<p>Imagine that we could write to <code>stdout</code> the port / socket on which we would like to open a communication with a new,
modern protocol, that would allow us to manipulate a new kind of “terminal.” Instead of using a regular text-based
terminal, it would be a piece of software allowing to manipulate pop-ups, text buffers, image / vector image buffers,
notification queues, etc. etc.</p>
<p>Obviously, I’m not inventing all that. If you know about <a href="https://www.gnu.org/software/emacs">emacs</a>, you know that that’s basically what they do, but
instead of using <code>stdin</code>, <code>stdout</code> or <code>execve</code>, they just evaluate Elisp. However, what I think could be very
interesting would be to allow native applications to do the same thing, with a native platform that would ship without
any real applications (so unlike <a href="https://www.gnu.org/software/emacs">emacs</a>), and let applications implement their own workflow.</p>
<p>That’s basically what I’ve been working on and experimenting lately. A platform that provides buffers, tree views,
notifications, pop-pups, overlays, etc. to allow any kind of applications that can read and write to sockets to use
those features as if they were native, and platform-independent, in the same way that you use <code>stdout</code> without even
thinking how it will be displayed in the terminal.</p>
<p>Leveraging such a platform would solve most of the problems I mentioned above, because:</p>
<ul>
<li>We could write a code / text editor that wouldn’t have to be worried about extensions like Git: the Git support would
come as a native application (equivalent of the CLI <code>git</code> application) that would open an overlay, for instance. That
application wouldn’t even know about the code editor: you would make that connection via the platform.</li>
<li>Applications  could communicate between each other to manipulate buffers, get the contents of the buffer and do
various kind of  operations on them (LSP servers, for instance).</li>
<li>Because the protocol would be completely rendering-agnostic, that would allow people to write an application that
would work for any kind of “platform implementations.”</li>
<li>Composition would use the low-level protocol to compose applications. You could imagine supporting an interface between
applications allowing to filter the contents of the buffer of one by using another application. Applications such as
<a href="https://github.com/junegunn/fzf">fzf</a> could <em>branch</em> on the buffers of a text editor, for instance. Or you could even open a buffer in the text editor
by running a fuzzy finder as another application. All of that would compose in a seamless way.</li>
<li>Etc. etc.</li>
</ul>
<p>Eh, of course, I know that such a platform protocol will probably never appear unless I make it, very opinionated and
subject to never be adopted by anyone else but me (or a small group of enthusiasts about that kind of ideas). Maybe this
blog article would have given you some ideas or hindsight about your own tools. Or maybe someone knows such a platform
(that I don’t) and will make me realize it already exists, or is a <em>work in progress</em>. Maybe, after all, it will just
launch a discussion about what <em>developing</em> means in 2022. Because to me, using tools that used to be efficient and
proved to be great ones a couple of years ago, doesn’t mean they are still a thing today. And I really mean it, because
I’m currently writing this very article in <a href="https://neovim.io">neovim</a>, in a <a href="https://sw.kovidgoyal.net/kitty">kitty</a> tab among dozens, and I just wish I had the DE I just
described above.</p>
<p>Keep the vibe!</p>
<blockquote>
<p><a href="https://www.reddit.com/r/programming/comments/vlp76z/development_environments_discussion_about">Reddit discussion</a></p>
</blockquote>


guid: https://strongly-typed-thoughts.net/blog/cargo-cyclic-dev-depsFri, 23 Aug 2019 12:00:00 GMT
title: cargo and how crates.io doesn’t like cyclic dev-dependencies
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/cargo-cyclic-dev-deps
feedurl: https://phaazon.net/blog/feed
content:
<p>This is a very short blog article about a problem I found lately with <a href="https://doc.rust-lang.org/cargo">cargo</a>.</p>
<h1>The problem</h1>
<p>Imagine you have a crate <code>A</code> and a crate <code>B</code> and you want to upload them to crates.io. There is a
context setup:</p>
<ul>
<li>You’re working in a <a href="https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html">cargo workspace</a>.</li>
<li><code>A</code> depends on <code>B</code> via <code>[dev-dependencies]</code> because you use it in your <code>examples/</code> directory,
for instance.</li>
<li><code>B</code> depends on <code>A</code> via <code>[dependencies]</code> because it implements some interface or such.</li>
<li>You would like to upload <code>A</code> first (because uploading <code>B</code> without having <code>A</code> would make <code>B</code>
unable to compile).</li>
</ul>
<p>The main problem is that currently, you cannot do <em>anything</em> with such a setup. If you try to
upload <code>A</code> to crates.io, crates.io will reject the crate because <code>B</code> doesn’t exist. That might be
stunning, but yes, <a href="https://doc.rust-lang.org/cargo">cargo</a> also check that your <code>[dev-dependencies]</code> are correctly aligned with your
<code>A</code>’s <code>Cargo.toml</code>. Meh.</p>
<p>If you try to upload <code>B</code>, obviously, since <code>A</code> is not there, crates.io will complain.</p>
<p>What should we do?</p>
<h1>The (dirty) solution</h1>
<p>First thing first, here’s <a href="https://github.com/rust-lang/cargo/issues/4242">a discussion</a> that helped
me figure out how to solve that issue. Disclaimer, though: it’s neither elegant and satisfying.</p>
<p>The first thing to do is to edit <code>A</code>’s <code>Cargo.toml</code> in order to completely strip its
<code>[dev-dependencies]</code> section. Just remove it. All of it.</p>
<p>Then, and that’s the tricky part: do not <code>git commit</code> the change. At that point, you have removed
the dependency from <code>A</code> to <code>B</code>, which is <strong>not mandatory for people getting the crate from
crates.io.</strong> That’s the nasty part. You’re about to upload a crate which metadata are not aligned
with your repository. I don’t like it, but I don’t see any other way to do it — however, you’ll see
we can fix it afterwards.</p>
<p>Then, upload <code>A</code> to crates.io:</p>
<pre><code>cd A
cargo publish --allow-dirty
</code></pre>
<p>The <code>--allow-dirty</code> switch is needed because <a href="https://doc.rust-lang.org/cargo">cargo</a> would complain about your working tree not
being clean otherwise.</p>
<p>Now, you have <code>A</code> on crates.io. Just go on with <code>B</code>: it can now be uploaded:</p>
<pre><code>cd ../B
cargo publish
</code></pre>
<p>Now you have, on crates.io:</p>
<ul>
<li><code>A</code> in version <code>X.Y.Z</code>, without any <code>[dev-dependencies]</code>.</li>
<li><code>B</code>, whichever version.</li>
</ul>
<p>The final part of the trick is to <em>restore</em> the sanity of <code>A</code>. Because, yes, I don’t like having
something on crates.io that is not exactly the same thing as my repository.</p>
<p>Edit <code>A</code>’s <code>Cargo.toml</code> or check it out:</p>
<pre><code>cd ..
git checkout .
</code></pre>
<p>Now you are exactly in the same situation as before trying anything to solve the problem. Edit
<code>A</code>’s <code>Cargo.toml</code> and increment its patch version. In my case, I would get version <code>X.Y.(Z+1)</code>.
<code>git commit</code> your changes, <code>git push</code>. You’re ready to patch crates.io:</p>
<pre><code>cd A
cargo publish
</code></pre>
<p>Now you have:</p>
<ul>
<li><code>A</code> in version <code>X.Y.(Z+1)</code>, with the right <code>[dev-dependencies]</code>.</li>
<li><code>B</code>, whichever version.</li>
</ul>
<h1>Conclusion</h1>
<p>I know it’s not an ideal fix, but at least we end up on our (clean) feets. I really really think
the <a href="https://doc.rust-lang.org/cargo">cargo</a> team should fix that issue, though. In that case, <code>A</code> and <code>B</code> were actually <a href="https://crates.io/crates/luminance">luminance</a>
and <a href="https://crates.io/crates/luminance-derive">luminance-derive</a>. I will make a long article very soon about them to introduce new versions,
new crates and complete graphics tutorials on how to have fun with <a href="https://crates.io/crates/luminance">luminance</a>.</p>
<p>I hope you liked that article and as always, <em>keep the vibes</em>.</p>


guid: https://strongly-typed-thoughts.net/blog/hop-trie-backtrack-fillingMon, 15 Mar 2021 00:00:00 GMT
title: Optimizing the permutation algorithm of hop.nvim
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/hop-trie-backtrack-filling
feedurl: https://phaazon.net/blog/feed
content:
<p>It’s been a while since I haven’t written something here. Today was a very exciting day to me as <a href="https://github.com/phaazon/hop.nvim">hop.nvim</a> has been
receiving more and more attention (and great contributions) from the <a href="https://neovim.io/">Neovim</a> community lately. I didn’t expect
people to adopt Hop that quickly and getting <em>actually</em> interested. So, thanks for all the feedback and the great
support!</p>
<p>Before starting up, if you don’t know what <a href="https://github.com/phaazon/hop.nvim">hop.nvim</a> is, here’s a small excerpt from the <code>README</code>:</p>
<blockquote>
<p>Hop is an EasyMotion-like plugin allowing you to jump anywhere in a document with as few keystrokes as possible.
It does so by annotating text in your buffer with hints, short string sequences for which each character represents
a key to type to jump to the annotated text. Most of the time, those sequences’ lengths will be between 1 to 3
characters, making every jump target in your document reachable in a few keystrokes.</p>
</blockquote>
<p>Today, I want to talk about something at the core of the design of Hop: permutations, and a new algorithm I implemented
to (greatly) optimize permutations in Hop.</p>
<blockquote>
<p>Disclaimer: it might be possible that the algorithm described here has an official name, as I’m using a well-known
data structure, traversing in a way that is also well established. Even though I came up with it, I don’t really know
whether it has a name so if you recognize it, or think it is similar to something, feel free to tell me!</p>
</blockquote>
<h1>Permutations</h1>
<p>When you run a Hop command (whether as a Vim command, such as <code>:HopWord</code>, or from Lua, like <code>:lua require'hop'.hint_words()</code>), Hop is going to place <em>labels</em>, called <em>hints</em>, in your window, over your buffer,
describing sequences of keys to type to jump to the annotated location in the buffer. Those key sequences are generated
as permutations of your <em>input key set</em>, of different lengths. Your input key set is basically the set of keys you
expect to type to hop around. For instance, the default key set is made for QWERTY and is:</p>
<pre><code>asdghklqwertyuiopzxcvbnmfj
</code></pre>
<p>The way Hop works is taking those keys and generating permutations of increasing length. The goal is to obviously type
as less as possible (remember: <em>Neovim motion on speed!</em>), so we want to generate 1-sequence permutations first, such as
<code>a</code>, <code>s</code>, <code>d</code>, etc., then switch to 2-sequence permutations, e.g. <code>aj</code>, <code>kn</code>, etc. The most important part of the work
done by Hop is to generate those permutations in a smart way.</p>
<p>Until today, I had implemented a single algorithm doing this. Let’s describe it so that I can introduce the actual topic
of this article.</p>
<h1>First permutation generation algorithm</h1>
<p>The first algorithm used to generate permutations in Hop was designed around the idea of being able to generate
permutations in a <a href="https://en.wikipedia.org/wiki/Corecursion">co-recursive</a> way (even though it’s not using co-recursion in Lua): given a permutation, you can
generate the next one by running the algorithm on the permutation. Doing that over and over generates more permutations.
You can sum it up like this:</p>
<pre><code class="language-lua">let first_perm = next_perm({})
let second_perm = next_perm(first_perm)
let third_perm = next_perm(second_perm)
-- etc. etc. stop at a given condition and return all the permutations
</code></pre>
<p>In functional programming languages, we call that kind of co-recursion an <em>unfold</em>. We stop building / generating when
meeting a condition. In our case, the condition is when we reach the number of permutations to generate. This first
algorithm had several constraints I wanted satisfied:</p>
<ol>
<li>It has to be fast and co-recursive, so that we can pass it the number of permutations to generate and simply generate
them until we have reached the number of desired permutations.</li>
<li>It has to take into account the user input key set.</li>
<li>It has to generate permutations minimizing the number of keys required to jump.</li>
<li>Because permutations will be distributed based on the <a href="https://en.wikipedia.org/wiki/Taxicab_geometry">Manhattan distance</a> to the cursor, they must be ordered /
sorted in a way that the first permutations in the list are the shorter ones and the last ones are the longer ones.</li>
</ol>
<p>We already saw <em>1.</em>, so let’s talk about the three remaining points. Taking into account the key set of the user, with
this algorithm, is done by splitting it into two sub-sets:</p>
<ul>
<li>A set called <em>terminal key set</em>, containing terminal keys. Those keys are used <em>only</em> in terminal position in
sequences — i.e. at the very right-most position in the sequence. For instance, in the sequence <code>abcde</code>, <code>e</code> is a
terminal key.</li>
<li>A set called <em>sequence key set</em>, containing sequence keys. Those are keys found as prefixes of terminal keys in
sequences. For instance, in the sequence <code>abcde</code>, <code>a</code>, <code>b</code>, <code>c</code> and <code>d</code> all appear before <code>e</code>, so they are sequence
keys.</li>
</ul>
<p>How to know which key is terminal and which is sequence? Well, I use a simple parameter for that, that can be modified
in the user configuration of Hop: <code>term_seq_bias</code> (<code>:help hop-config-term_seq_bias</code> if you are interested). It is a
floating number parameter specifying the ratio of terminal vs. sequence keys to use. Imagine you have 100 keys in your
key set (you typically will have between 20 and 30), setting this parameter to <code>3 / 4</code> makes it use 75 terminal keys and
25 sequence keys.</p>
<p>Then, given those two sets, we start from the 0-sequence permutation, a.k.a. <code>{}</code>, and we start using terminal keys. If
we have <code>abc</code> as terminal keys and <code>de</code> as sequence keys, we will get the permutations <code>'a'</code>, <code>'b'</code> and <code>'c'</code> for
<code>n = 3</code>. If we ask permutations for <code>n = 4</code>, we will get <code>'a'</code>, <code>'b'</code>, <code>'c'</code> but not <code>'d'</code>, as it’s a sequence key.
Instead, we will start a new layer by incrementing the dimension of sequences, yielding 2-sequence permutations: we will
then generate <code>'da'</code>. For <code>n = 6</code>, the last permutation in the list is <code>'dc'</code>, which means that for <code>n = 7</code>, we have to
do something, as we have run out of terminal keys. Instead of starting a new layer (3-sequence permutations), we
traverse the permutation in reverse order, checking that we have exhausted all the sequence keys. Here, we can see that
<code>'d'</code> is not the last sequence key, so we can use the next one, <code>e</code>, and start a new sequence at <code>ea</code>, then <code>eb</code>, <code>ec</code>…
and guess what permutation is the next? Since we have run out of both terminal keys and sequence keys, we need to use
3-sequence keys: the next permutation will be <code>dda</code>.</p>
<p>This algorithm is interesting because it allows people to <em>bias</em> (hence the name) the distribution of terminal and
sequence keys. However, it is pretty obvious that this algorithm does a pretty poor job at minimizing the overall number
of keys to type: it will minimize the number of keys to type for short hops, mostly around the cursor and at
mid-distance. I highly advise to use either <code>3 / 4</code> or even <code>0.5</code> for the bias value. Other values yield… weird results.</p>
<h1>The problem and finding a solution</h1>
<p>Very quickly, I have noticed something a bit annoying with Hop. Even though it is pretty fast, the distribution of keys
for long jumps is often using 3-sequence permutations. And where does it make sense to use Hop the most? For mid to long
range jumps. Vim and Neovim are already pretty good at short jumps. For instance, jumping to characters on the same line
can be achieved with <kbd>f</kbd> and <kbd>F</kbd>. If you want to jump to a word on the line above, you can just press
<cmd>k</cmd> and use horizontal jumps.</p>
<p>I don’t necessarily <em>agree</em> that those are always ideal (and, well, I do use Hop for short hops too), but the situation
is not ideal for long jumps in Vim / Neovim. See, I’ve been using Vim and Neovim for more than 12 years now, and I’ve
seen lots of people using it. Most of the time, what people use for long jumps is either (or a combination of) one of
these:</p>
<ul>
<li>Turn on <code>relativenumber</code> and look at relative numbers to jump to the line where the location appears in, press
something like <code>17k</code> or <code>17j</code>, then use the <kbd>f</kbd> / <kbd>F</kbd> motions.</li>
<li>Use the real line number and commands such as <code>:153</code> or <code>153gg</code> to jump to line 153, and do the same as the previous
point.</li>
<li>Use <kbd>{</kbd> / <kbd>{</kbd> / <kbd>%</kbd> to quickly skip large portions of text and arrive at destination in
less typing effort (I’m actually a big user of <kbd>%</kbd>).</li>
<li>Use marks, LSP, tags etc. to directly jump to semantic locations.</li>
</ul>
<p>Among all those options, even when you combine them, you will always have to type more keys / take more time to do
long jumps. For instance, considering we can jump with <kbd>f</kbd> to our location <code>l</code> because it is unique on the line we
want to go to, assuming it’s on line 1064, 45 lines above our cursor:</p>
<ul>
<li>With <code>relativenumber</code>, we have to press <code>45kfl</code>: between 5 and 6 keys (depending on whether you have to press shift on
your keyboard layout for the digits).</li>
<li>With real line number, <code>1064ggfl</code>, between 8 and 9 keys.</li>
</ul>
<p>So clearly, Hop should help for those jumps as much as possible, and the previous algorithm, even though already an
enhancement over typing relative numbers or real line numbers, can still be enhanced. It is already pretty good because
it will provide you with, most of the time, between 1-sequence and 3-sequence permutations. For long jumps, assume the
longest: 3 keys, plus one key to trigger the Hop mode, you get 4 keys to type at most to jump anywhere.</p>
<p>The goal is to reduce that to 2 keys, i.e. 2-sequence at most, most of the time — 3 keys if you count the key to start
Hop. In order to understand the concept of this new algorithm, let’s focus on what was wrong with the former. Consider
the following, using <code>abc</code> as terminal keys and <code>de</code> as sequence keys:</p>
<pre><code class="language-lua">-- n = 3
a b c

-- n = 4
--  last 1-sequence
--  v
a b c da

-- n = 6
a b c da db dc

-- n = 9
a b c da db dc ea eb ec

-- n = 10
--                   last 2-sequence
--                   v
a b c da db dc ea eb ec dda
</code></pre>
<p><img src="https://phaazon.net/media/uploads/hop_term_seq_bias_showcase.png" alt="" /></p>
<p>You can see that after only 9 words, we are already using 3-sequence permutations. What about, for instance, the
following sequences:</p>
<pre><code class="language-lua">aa ab ac bb dd de …
</code></pre>
<p>Obviously, because of the rules explained above about the difference between <em>terminal</em> and <em>sequence</em> keys, those are
forbidden: indeed, they would yield <em>ambiguous</em> sequences. For instance, <code>aa</code> and <code>a</code> both start with the same key and
one is terminal at a given depth (1-sequence) while the other still has one level (2-sequence): what should we do?
Trigger the 1-sequence and never be able to jump to the 2-sequence? Use a timeout so that we have the time to type the
second <code>a</code>? But that would make jumping to terminal keys feel delayed / slow, so that’s a hard no. What’s the
problem?</p>
<p>The problem is that we are not efficiently using the key space by forbidding those sequences. If you pay attention, you
should actually be able to use <em>all possible 2-sequence permutations</em>. But in order to do that… we have to forbid using
1-sequence permutations! Forbidding that will make all 2-sequence permutations unique and unambiguous. The difference is
massive: instead of having only 9 permutations shorter than 3-sequence ones, we know have 25, which is the number of
keys in the user key set squared: <code>'abcde'</code> is 5 keys, and 5² is 25. For the default key set for QWERTY, which has 26
keys, it means 26² = 676 maximum 2-sequence permutations, which is a comfortable number for the visible part of your
buffer.</p>
<h1>The new algorithm: tries and backtrack filling</h1>
<p>The new algorithm’s idea relies on using all the keys from your input key set. It doesn’t split it into terminal and
sequence keys. It creates <a href="https://en.wikipedia.org/wiki/Trie">tries</a> (a type of search tree, optimized for packed storage and fast traversal) to store the
permutations and backtracks to remove ambiguity as we ask for more permutations. For instance, using the user input key
set <code>'abcde'</code>:</p>
<pre><code class="language-lua">-- n = 5
a b c d e

-- n = 6
--      e was transformed to ea
--      v
        a b
a b c d e e

-- n = 7
        a b c
a b c d e e e

-- n = 9
        a b c d e
a b c d e e e e e
</code></pre>
<p>As you can see, for <code>n = 6</code>, instead of adding a new layer, the algorithm backtracks to fill what is possible to fill:
in our case, <code>e</code>, by replacing it with <code>ea</code> and inserting the new permutation at <code>eb</code>. Once a given layer is saturated,
the algoritm backtracks again, trying to saturate another layer:</p>
<pre><code class="language-lua">-- n = 10
--    d was transformed to da
--    v
      a b a b c d e
a b c d d e e e e e

-- n = 13
      a b c d e a b c d e
a b c d d d d d e e e e e

-- n = 17
    a b c d e a b c d e a b c d e
a b c c c c c d d d d d e e e e e

-- n = 21
  a b c d e a b c d e a b c d e a b c d e
a b b b b b c c c c c d d d d d e e e e e

-- n = 25
a b c d e a b c d e a b c d e a b c d e a b c d e
a a a a a b b b b b c c c c c d d d d d e e e e e
</code></pre>
<p>Here, we have completely saturated the 2-sequence permutations, yielding 25 of them, and cannot backtrack anymore. When
backtracking fails, we simply augment the last trie, deepest:</p>
<pre><code class="language-lua">-- n = 26
                                                a b
a b c d e a b c d e a b c d e a b c d e a b c d e e
a a a a a b b b b b c c c c c d d d d d e e e e e e
</code></pre>
<p>And we go on. Here, you can see a repetition of the same key, such as <code>e e e e</code>, but because those are tries, they are
going to be actually encoded as a single node, having several children. Consider this example:</p>
<pre><code class="language-lua">-- encode these permutations
--
--         a b c d
-- a b c d e e e e

local trie = {
  {
    key = 'a';
    trie = {}
  },
  {
    key = 'b';
    trie = {}
  },
  {
    key = 'c';
    trie = {}
  },
  {
    key = 'd';
    trie = {}
  },
  {
    key = 'e';
    trie = {
      { key = 'a'; trie = {} },
      { key = 'b'; trie = {} },
      { key = 'c'; trie = {} },
      { key = 'd'; trie = {} },
    }
  }
}
</code></pre>
<p>Two interesting properties about this algorithm:</p>
<ul>
<li>It does distribute permutations more equitably across your buffer as the number of permutations to generate grows, but
will do it from the end of tries, making the first permutations the shortest as long as possible.</li>
<li>When tries are saturated, we can lose 1-sequence permutations, but such a scenario authorizes to jump anywhere with
only 2-sequence permutations.</li>
</ul>
<p><img src="https://phaazon.net/media/uploads/hop_trie_backtrack_filling_showcase.png" alt="" /></p>
<h1>Conclusion</h1>
<p>This new algorithm was a lot of fun to come up with, design, test, implement and eventually use, because yes, it is
already available (as a default) in <a href="https://github.com/phaazon/hop.nvim">hop.nvim</a>. If for any reason you would prefer to use the first one, you can still
change that in the user configuration (see <code>:h hop-config-perm_method</code>). Another point, too: since the support for the
<code>setup</code> workflow, it’s been possible to locally override Lua function calls <code>opts</code> argument. You can then make a key
binding using the first algorithm and another one using the new one – all this is explained in the vim help page 😉.
It’s up to you!</p>
<p>Keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/warmy-0.6.0Sat, 24 Feb 2018 19:46:00 GMT
title: warmy-0.6.0; bug fixes and rewrite
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/warmy-0.6.0
feedurl: https://phaazon.net/blog/feed
content:
<h1>warmy-0.6.0</h1>
<p>Today I released a new version of <a href="https://crates.io/crates/warmy">warmy</a>, the <code>warmy-0.6.0</code> release. That release kicks in with a
few additions, among:</p>
<ul>
<li>Complete rewrite of its internals, enabling optimizations (mostly about allocations).</li>
<li>A nasty IO-bound bug was removed.</li>
</ul>
<p>That bug caused long-lasting reloads on stream-copied resources go into weird behavior.</p>
<h1>The bug</h1>
<p>In order to get the bug, I must give a bit of context.</p>
<p>Imagine you have a large resource, like a 4K texture or a big mesh. Whenever your favourite
software writes it to disk, it’s very likely it’ll <em>stream</em> chunks by chunks of bytes. For
instance, it might choose to copy the resource <em>2 MB</em> by <em>2 MB</em> on disk. On each copy, your
file system will generate <code>WRITE</code> events, that <a href="https://crates.io/crates/warmy">warmy</a> will intercept. Before <code>warmy-0.6.0</code>, the
default behavior was to reload the resource on the first <code>WRITE</code> event, which was already
wrong, because only a very small part of the resource would have changed – I actually witnessed
weird behaviors with textures in a demo of mine I’m working on, not seeing the new textures in my
demo.</p>
<p>But there’s worse. There’s a parameter you can set of your <code>Store</code>, called <code>update_await_time_ms</code>.
That parameter gives <a href="https://crates.io/crates/warmy">warmy</a> a hint about how much time must have passed since the last update in
order to effectively call the <a href="https://docs.rs/warmy/0.5.2/warmy/trait.Load.html#method.reload"><code>Load::reload</code></a> function. However, this is a bit twisted, because if
the resource takes more time to reload than <code>update_await_time_ms</code>, it’ll get repeatedly loaded –
for as many as <code>WRITE</code> events were generated. This is a bit sick, yeah.</p>
<h1>The fix</h1>
<p>The fix was pretty simple: change the semantics of that <code>update_await_time_ms</code>. In <code>0.5.2</code>, it has
the default value of <strong>1s</strong>, meaning that a resource wouldn’t reload if it was reloaded less than a
second ago. The new semantics works on the future. Whenever a <code>WRITE</code> event is intercepted, <a href="https://crates.io/crates/warmy">warmy</a>
will call the <a href="https://docs.rs/warmy/0.5.2/warmy/trait.Load.html#method.reload"><code>Load::reload</code></a> function only if no <code>WRITE</code> event is intercepted in the next
<code>update_await_time_ms</code>. It’s a bit like the implementation of a click and a double click: you must
wait a bit after you got a <code>MouseRelease</code> event in order to interpret is as a <code>Click</code> because
another <code>MouseRelease</code> could arrive soon (generating a <code>DoubleClick</code> if it’s soon enough).</p>
<blockquote>
<p>You’ll also notice that <code>update_await_time_ms</code> name sticks better to the new semantics!</p>
</blockquote>
<p>In <code>0.5.2</code>, the default value for <code>update_await_time_ms</code> was <strong>1s</strong>. If we kept that value, it would
result in a pretty bad overall <em>latency</em>. The value was lowered to <strong>50ms</strong> instead.</p>
<blockquote>
<p>You can still tweak that value if it doesn’t suit your needs.</p>
</blockquote>
<p>More information can be found in the <a href="https://github.com/phaazon/warmy/blob/master/CHANGELOG.md#060">changelog</a>.</p>
<h1>On the future of the crate</h1>
<p>I’ve been very happy with what <a href="https://crates.io/crates/warmy">warmy</a> has brought to me so far. Other people also gave it a try
and for now seem to enjoy it. I’ve gathered a few ideas for the future, based on IRL talks over 
<strike>a beer</strike> several beers, and GitHub issues / pull requests:</p>
<ul>
<li>We want to be able to pass around a <em>context</em> when loading and reloading. This would enable
tweaking the loading behavior of the objects and pass values from the calling function. However,
even though the feature is well-identified, it is not well-defined in <a href="https://crates.io/crates/warmy">warmy</a> and more research
must be done.</li>
<li>Multithreading. There’re a few ideas I’d like to implement, like an IO thread that would do all
the file IO-bound computation and dispatch the bytes to the other threads; and using <code>Future</code>.</li>
</ul>
<p>Feel free to test it, and as always, keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/dont_use_defaultThu, 16 Jul 2015 00:00:00 GMT
title: Don’t use Default
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/dont_use_default
feedurl: https://phaazon.net/blog/feed
content:
<h1>Don’t use Default</h1>
<h2>Background knowledge: typeclasses and laws</h2>
<p>In <strong>Haskell</strong>, we have a lot of
<a href="https://en.wikipedia.org/wiki/Type_class">typeclasses</a>. Those are very handy
and – in general – come with laws. Laws are very important and give hints on how
we are supposed to use a <em>typeclass</em>.</p>
<p>For instance, the <code>Semigroup</code> <em>typeclass</em> exposes an operator (<code>(&lt;&gt;)</code>) and has
an <strong>associativity</strong> law. If <code>a</code>, <code>b</code> and <code>c</code> have the same type <code>T</code>, if we know
that <code>T</code> is a <code>Semigroup</code>, we have the following law:</p>
<pre><code>a &lt;&gt; b &lt;&gt; c = (a &lt;&gt; b) &lt;&gt; c = a &lt;&gt; (b &lt;&gt; c)
</code></pre>
<p>If <code>T</code> is a <code>Monoid</code>, which is a <code>Semigroup</code> with an identity (called <code>mempty</code>),
we have a law for <em>monoids</em>:</p>
<pre><code>a &lt;&gt; mempty = mempty &lt;&gt; a = a
</code></pre>
<p>Those laws can – <em>have</em> – to be used in our code base to take advantage over the
structures, optimize or avoid boilerplate.</p>
<h2>The Default typeclass</h2>
<p>In some situations, we want a way to express default values. That’s especially
true in OO languages, like in the following <strong>C++</strong> function signature:</p>
<pre><code>void foo(float i = 1);
</code></pre>
<p>In <strong>Haskell</strong>, we cannot do that, because we have to pass all arguments to a
function. The following doesn’t exist:</p>
<pre><code>foo :: Float -&gt; IO ()
foo (i = 1) = -- […]
</code></pre>
<p>And there’s more. Even in C++, how do you handle the case when you have several
arguments and want only the first one to be defaulted? You <strong>cannot</strong>.</p>
<p>So, so… Some <strong>Haskellers</strong> decided to solve that problem with a <em>typeclass</em>.
After all, we can define the following typeclass:</p>
<pre><code class="language-haskell">class Default a where
  def :: a
</code></pre>
<p>We can then implement <code>Default</code> and have a <em>default</em> value for a given type.</p>
<pre><code class="language-haskell">instance Default Radians where
  def = Radians $ 2*pi

instance Default Fullscreen where
  def = Fullscreen False
</code></pre>
<p>However, there is an issue with that. You cannot use <code>Default</code> without creating
<code>newtype</code>s to overload them. Why? Well, consider the following <code>Default</code>
instance:</p>
<pre><code class="language-haskell">instance Default Float where
  def = -- what should we put here?
</code></pre>
<p>Remember that, in <strong>Haskell</strong>, an instance is defined only once and is
automatically imported when you import the module holding it. That means you
cannot have the following:</p>
<pre><code class="language-haskell">-- in a module A
instance Default Float where
  def = 0

-- in a module B
instance Default Float where
  def = 1

-- in a module C
import A
import B

-- What instances should we use?
</code></pre>
<blockquote>
<p><em>Hey, that’s easy. We just have to keep the modules apart, and import the one
we want to use!</em></p>
</blockquote>
<p>Yeah, well. No. No.</p>
<p><img src="http://phaazon.net/pub/god_please_no.gif" alt="" /></p>
<p>Orphan instances are <strong>wrong</strong>. You should read
<a href="https://wiki.haskell.org/Multiple_instances">this</a> for further explanations.</p>
<p>That’s why we have to use <code>newtype</code>s everywhere. And that’s boring. Writing code
should always have a goal. When we write as much boilerplate code as real code,
we can start thinking there’s something wrong. Worse, if we have more
boilerplate than real code, well, something is terribly wrong. In our case,
we’re introducing a lot of <code>newtype</code>s for only being able to use <code>def</code> at a few
spots. Is that even worth it? Of course not.</p>
<h2>Default is evil</h2>
<p>The <code>Default</code> typeclass is evil. It’s shipped with default instances, like one
for <code>[a]</code>, which defaults to the empty list – <code>[]</code>. It might be clear for you
but why would I want to default to the empty list? Why not to <code>[0]</code>? Or a more
complex list? I really doubt someone ever uses <code>def :: [a]</code>.</p>
<p>Another reason why <code>Default</code> is wrong? There’s absolutely <strong>no law</strong>. You just
have a default value, and that’s all.</p>
<pre><code class="language-haskell">bar :: (Default a) =&gt; a -&gt; Maybe String
</code></pre>
<p>Can you say what the default is for? Of course you cannot. Because there’s no
law. The instance has no real meaning. A default value makes sense only for the
computation using it. For instance, the empty list makes sense if we glue it to
the <em>list concatenation</em>.</p>
<h2>We already have a lot of defaults</h2>
<p>In <a href="https://hackage.haskell.org/package/base">base</a>, there’re already several
ways to express <em>defaulted values</em>.</p>
<p>The <code>Monoid</code>’s <code>mempty</code> is a way to express a default value regarding its
binary operation (<code>(&lt;&gt;)</code>).</p>
<p>The <code>Alternative</code>’s <code>empty</code> provides a similar default, but for first-class
types.</p>
<p>The <code>MonadZero</code>’s <code>mzero</code> provides a different default, used to <em>absorb</em>
everything. That’s a law of <code>MonadZero</code>:</p>
<pre><code>mzero &gt;&gt;= f = mzero
a &gt;&gt; mzero  = mzero
</code></pre>
<p>Those defaults are interesting because we can reason about. If I see <code>mzero</code> in
a <em>monadic code</em>, I know that whatever is following will be discarded.</p>
<h1>Conclusion</h1>
<p>So please. Stop using <code>Default</code>. It doesn’t bring any sense to your codebase. It
actually removes some! If you want to provide functions with defaulted
arguments, consider partially applied functions.</p>
<p><img src="http://phaazon.net/pub/default_stats.png" alt="" /></p>
<p><a href="https://hackage.haskell.org/package/data-default">data-default</a> is a very
famous package – look at the downloads! You can now have some hindsight about it
before downloading it and ruining your design. ;)</p>
<p>Happy hacking. :)</p>


guid: https://strongly-typed-thoughts.net/blog/luminance_first_tutorialThu, 24 Sep 2015 00:00:00 GMT
title: luminance first tutorial
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance_first_tutorial
feedurl: https://phaazon.net/blog/feed
content:
<h1>Woah!</h1>
<p>I’m very happy about people getting interested about my <a href="http://hackage.haskell.org/package/luminance">luminance</a>
graphics framework. I haven’t received use case feedback yet, but I’m pretty confident I will sooner
or later.</p>
<p>In the waiting, I decided to write an <em>embedded tutorial</em>. It can be found
<a href="http://hackage.haskell.org/package/luminance-0.1.1/docs/Graphics-Luminance.html">here</a>.</p>
<p>That tutorial explains all the basic types of luminance – not all though, you’ll have to dig in the
documentation ;) – and describes how you should use it. I will try to add more documentation for
each modules in order to end up with a very well documented piece of software!</p>
<h1>Let’s sum up what you need</h1>
<p>People on <a href="https://www.reddit.com/r/haskell/comments/3lyxzc/luminance_01_released">reddit</a>
complain – they are right to – about the fact the samples just <em>“didn’t work</em>. They actually did,
but the errors were muted. I released
<a href="http://hackage.haskell.org/package/luminance-samples-0.1.1">luminance-0.1.1</a> to fix that issue. Now
you’ll get the proper error messages.</p>
<p>The most common issue is when you try to run a sample without having the required hardware
implementation. luminance requires <strong>OpenGL 4.5</strong>. On Linux, you might need to use <code>primusrun</code> or
<code>optirun</code> if you have the <strong>Optimus</strong> technology. On Windows, I guess you have to allow the samples
to run on the dedicated <em>GPU</em>. And on Mac OSX… I have no idea; <code>primusrun</code> / <code>optirun</code>, I’d go.</p>
<p>Anyways, I’d like to thank all people who have/will tried/try the package. As always, I’ll keep you
informed about all the big steps I take about luminance. Keep the vibe!</p>


guid: https://strongly-typed-thoughts.net/blog/note-taking-2023Thu, 12 Oct 2023 20:35:00 GMT
title: Reflections about note-taking
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/note-taking-2023
feedurl: https://phaazon.net/blog/feed
content:
<p>TL;DR I was wrong about <a href="https://orgmode.org/">org-mode</a>. I know, this is a bit click-bait, but really, I made wrong assumptions and I think I
need to explain my thoughts now.</p>
<h1>Note taking, task management, journaling… how?</h1>
<p>As an experienced Software Engineer, I know that I need to multi-task and work on many different projects in parallel,
with different chronicity. Some new tasks can be done only a couple of minutes after discovering / creating them, and
some others will span on days, weeks if not months. That leads to having many parallel things on-going, and since I want
to do my job as correctly as possible… I take notes. I take notes about pretty much <em>everything</em>, and I’ve been using
note-taking apps more and more.</p>
<p>Years ago, as I discovered <a href="https://github.com/doomemacs/doomemacs">Doom Emacs</a>, I also discovered <a href="https://orgmode.org/">org-mode</a>. I was blown away by the features brought by
<a href="https://github.com/doomemacs/doomemacs">Doom Emacs</a>, and a bit puzzled with other things. For instance, the agenda (I rarely care about ETA for my personal
notes; that’s something that is tracked at the company-wide level, for instance in the infamous Jira or whatever you’re
using for project management in your team — and I just don’t care on my spare-time). Also, I was a bit doubtful about
having notes laid as plain text (yes, <a href="https://orgmode.org/">org-mode</a> formatted, but still text). What it means is that, there is no database
nor state. Everytime you want to get a list of your <code>TODO</code>, <a href="https://orgmode.org/">org-mode</a> has to go through all your <code>.org</code> files and
generates the list for you. Back then, I found that a bit weird, but today, I have changed my mind.</p>
<h1>Zettelkasten and other tools</h1>
<p>The way I take notes, whatever I do, is to minimize the note files I create. For instance, at work, I have many notes
about commands to run with a specific product. Instead of having a note <code>That product</code>, I tend to create smaller notes,
more scoped, like <code>That product - deploy</code>, <code>That product - how to configure</code>, etc. That allows two important things:</p>
<ul>
<li>Notes are shorter, easier to read and maintain.</li>
<li>Looking up for information and fuzzy searching is easier.</li>
</ul>
<p>Yes, <em>fuzzy searching</em>. Whatever the tool I use, I often need to search for notes, and either I search by note names, or
by content.</p>
<p>A couple of months ago, I wrote <a href="https://github.com/phaazon/mind">mind</a>, a rewrite of <a href="https://github.com/phaazon/mind.nvim">mind.nvim</a>. I designed that system to be a <em>tree editor</em>, in which
nodes are either “document node” containing some metadata and a path to a file, or a “link node”, with metadata and URL.
Those tools were very useful to me, and I’ve been using them for a long time.</p>
<p>However, as I moved more and more into the UNIX world with <a href="https://kakoune.org">kakoune</a> and <a href="https://github.com/phaazon/kak-tree-sitter">kak-tree-sitter</a>, I realized that I should
maybe revisit the way I think about notes. For instance, what I often need is to list the on-going tasks I’m doing,
and updating them. Oftentimes (if not all the time), tasks emerged from a context, like a discussion with a colleague,
some notes / brainstorming with myself I’m doing / a meeting / etc. And <a href="https://github.com/phaazon/mind">mind</a>, <a href="https://www.notion.so/">Notion</a> — or whatever that is using
an external state — are not very suited. The problem relies in <em>interruption</em>. You’re in the editor of your choice,
and then you have to switch to another window, look for the task / notes, use some keybindings to change the task to
switch it from <code>TODO</code> to <code>WIP</code> or <code>WIP</code> to <code>DONE</code> or <code>WONTDO</code> or whatever.</p>
<p>As I was working more and more with <a href="https://github.com/phaazon/mind">mind</a>, I realized this kind of workflow was starting to annoy me.</p>
<h1>Composing</h1>
<p>You start to see the pattern with my blog posts: I enjoy the UNIX world <em>a lot</em>. And even though <a href="https://github.com/phaazon/mind">mind</a> is perfectly
fine regarding that (I was even able to capture notes by using <a href="https://github.com/tmux/tmux">tmux</a>’’s <code>display-popup</code> with <a href="https://github.com/phaazon/mind">mind</a>), I wanted to
switch to something… easier, yet still UNIX.</p>
<p>Then I started to think again about <a href="https://orgmode.org/">org-mode</a>. See, <a href="https://orgmode.org/">org-mode</a> is eventually just a format specification. The Emacs
mode is the reference implementation, with major modes, keybindings etc. but if Emacs can edit and do stuff to <code>.org</code>
files, then the concept is transposable to other tools. The question is: what is the core of <a href="https://orgmode.org/">org-mode</a>? The idea
behind <a href="https://orgmode.org/">org-mode</a> is that there is no state: the notes themselves are the state. There is no need for an external tool.
Any tool can open an <code>.org</code> file and edit it. Yes, convenience is better to navigate <code>.org</code> notes, but that’s just a
matter of tooling. If you want to open and edit an <code>.org</code> file using your favorite editor, you should be able to.</p>
<p>Then, for the rest, <a href="https://orgmode.org/">org-mode</a> has a couple of interesting and useful concepts:</p>
<ul>
<li>You write notes in text files, including metadata. So tasks management is also done via <code>.org</code> files.</li>
<li>Listing tasks, for instance, can be performed pretty easily by grepping headings. For instance, <code>* TODO</code> can be used
at the beginning of a line to start a TODO item, that will appear in the listing. That looks arrogantly innocent, but
it’s really powerful, and I’ll explain why just below.</li>
<li><a href="https://orgmode.org/">org-mode</a> has the concept of a <em>capture</em> buffer. The idea is simple: you open it to add stuff right off your mind,
and you come back to it later to triage its content. This is a fantastic concept, because very often, I need to take
notes about things I have to do, but I have no structure yet. For instance,
<code>* TODO talk to Bob about service-epsilon</code>. Later on, I might create a note <code>Epsilon</code> and move the <code>TODO</code> item into
the note. Since there is no state, the <code>TODO</code> will still appears in my todo-list.</li>
<li>Another important thing is <em>archiving</em>. You are encouraged to create many small notes, even for task management. At
work, I currently have a note called <code>Back from holiday Oct 2023</code>, which contains many <code>TODO</code> items taken from the
backlog I read on Slack (basically, stuff my colleagues did and that I want to catch up with). At some point, I will
be done with all of that, so instead of keeping <code>DONE</code> notes around (that will show up in my <code>DONE</code> listing), I can
just archive that note. It will not appear anymore in my fuzzy search results.</li>
</ul>
<p>All of that is really exciting, because it makes note taking easy and seamless; it doesn’t require any state, and you
can pick the tools you want to implement the workflow.</p>
<h1>It’s not a tool; it’s a workflow</h1>
<p>Yes, that’s what I think is the most important thing here. I do not want to use <a href="https://orgmode.org/">org-mode</a> for now. I like Markdown.
Even though I agree <code>.org</code> files are more powerful, Markdown, at least for now, is more than enough to me. However, I
wanted to augment it to do things slightly similar to <a href="https://orgmode.org/">org-mode</a>.</p>
<p>And so I did it. I created in my <a href="https://kakoune.org">kakoune</a> a couple of commands to:</p>
<ul>
<li>Open a capture buffer located at the same place on my disk. It defaults to <code>~/notes/capture.md</code>.</li>
<li>Quickly capture a line via <a href="https://kakoune.org">kakoune</a>’’s user prompt and append it to the capture buffer (super useful for todo items).</li>
<li>Create a new note in <code>~/notes/notes/</code>.</li>
<li>Archive a note into <code>~/notes/archives/</code>.</li>
<li>List / fuzzy search notes.</li>
<li>List / fuzzy search archives.</li>
<li>Open a daily journal in <code>~/notes/journal/&lt;year&gt;/&lt;month&gt;/&lt;day&gt;.md</code>. It’s super easy with <a href="https://kakoune.org">kakoune</a> since you have
<code>%sh{}</code> expansion. Just use the <code>date</code> command you’re done!</li>
<li>List daily journals.</li>
<li>Open task list by:
<ul>
<li><code>TODO</code>, <code>WIP</code>, <code>DONE</code> or <code>WONTDO</code> items.</li>
<li>Those items are put into a special buffers on which I have set <code>buffer</code> keybindings to jump to the place where the
item is located. For instance, if I list <code>TODO</code> items, pressing <code>&lt;ret&gt;</code> on a line will open the appropriate file and
move my cursor to the right line and column where the <code>TODO</code> item is defined.</li>
<li>Labels, like <code>:learn:</code> or <code>:hard:</code> or whatever.</li>
</ul>
</li>
<li>Local (<code>buffer</code>) keybindings to switch tasks to different status.</li>
<li>A special convenience I created: Markdown files containing a special <code>&lt;!-- github_project: &lt;user&gt;/&lt;project&gt; --&gt;</code>,
usually at the end of the file, allows me to press a keybinding on issue numbers (like <code>#123</code>) to automatically open
them in my browser.</li>
<li>Synchronize notes with a single keybinding, which is running some <code>git</code> commands to rebase / push.</li>
<li>And many more similar features.</li>
</ul>
<blockquote>
<p>I plan on releasing the <code>.kak</code> file containing all of that if people are interested.</p>
</blockquote>
<p>The great thing is that most of the features, like fuzzy searching, are implemented via external tools. I use <code>fd</code> and
<code>rg</code> fuzzy search todo items. There is nothing to do in <a href="https://kakoune.org">kakoune</a> besides gluing everything together.</p>
<p>This setup has been my daily driver for months now and I really enjoy it. Being able to take notes and just slide in a
<code>- TODO I need to check that</code> right in the middle of the file and having it appear in a listing later is a really
powerful aspect of this workflow… and <a href="https://orgmode.org/">org-mode</a> was doing that all along ever since. Eh, it almost makes me nostalgic
of my time using the Emacs plugin!</p>
<p>I hope that gave me some (not so fresh) hindsight about note taking and task management (and journaling!). Have fun
hacking around!</p>


guid: https://strongly-typed-thoughts.net/blog/pure_vs_ioThu, 18 Feb 2016 00:00:00 GMT
title: Pure API vs. IO-bound API for graphics frameworks
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/pure_vs_io
feedurl: https://phaazon.net/blog/feed
content:
<p>Hi! It’s been a while I haven’t posted something here. I haven’t been around for a few weeks and
I’ve been missing writing here. I didn’t work that much on
<a href="https://www.stackage.org/package/luminance">luminance</a> or other projects, even though I still
provided updates for stackage for my packages. I worked a bit on
<a href="https://github.com/phaazon/cheddar">cheddar</a> and I hope to be able to release it soon!</p>
<p>Although I didn’t add things nor write code at home, I thought a lot about graphics API designs.</p>
<h1>Graphics API designs</h1>
<h2>Pure API</h2>
<p>APIs such as <a href="http://www.lambdacube3d.com/">lambdacube</a> or
<a href="https://hackage.haskell.org/package/GPipe">GPipe</a> are known to be graphics <em>pure API</em>. That means
you don’t have use functions bound to <code>IO</code>. You use some kind of
<a href="http://www.haskellforall.com/2012/06/you-could-have-invented-free-monads.html">free monads</a> or
an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">AST</a> to represent the code that will run on
the target GPU. That pure design brings numerous advantages to us:</p>
<ul>
<li>it’s possible to write the GPU code in a declarative, free and elegant way;</li>
<li>because of not being <code>IO</code>-bound, side-effects are reduced, which is good and improves the code
safety;</li>
<li>one can write GPU code and several interpreters or/and with different technologies, hence a
loosely coupled graphics API;</li>
<li>we could even imagine serializing GPU code to send it through network, store it in a file or
whatever.</li>
</ul>
<p>Those advantages are nice, but there are also drawbacks:</p>
<ul>
<li>acquiring resources might not be explicit anymore and none knows exactly when sparce resources
will be loaded into memory; even though we could use <em>warmup</em> to solve that, <em>warmup</em> doesn’t
help much with dynamic scene where some resources are loaded only if the context enables it (like
the player being in a given area of the map, in a game, for instance);</li>
<li>interfacing! people will have to learn how to use free monads, AST and all that stuff and how to
interface it with their own code (<em>FRP</em>, render loops, etc.);</li>
<li>performance; even though it should be okay, you’ll still hit a performance overhead by using pure
graphics frameworks, because of internals mechanisms used to make it work.</li>
</ul>
<p>So yeah, a pure graphics framework is very neat, but keep in mind there’s – so far – no proof it
actually works, scales nor is usable for a decent high-performance for end-users. It’s the same
dilemna as with Conal’s <em>FRP</em>: it’s nice, but we don’t really know whether it works <em>“at large scale
and in the real world”</em>.</p>
<h2>IO-bound API</h2>
<p>Most of the API out there are IO-bound. <em>OpenGL</em> is a famous C API known to be one of the worst one
on the level of side-effects and global mutations. Trust me, it’s truly wrong. However, the pure
API as mentioned above are based on those impure IO-bound APIs. So we couldn’t do much without them.</p>
<p>There are side effects that are not that bad. For instance, in OpenGL, creating a new buffer is a
side-effect: it requires that the CPU tell the GPU <em>“Hey buddy, please create a buffer with that
data, and please give me back a handle to it!”</em>. Then the GPU would reply <em>”No problem pal,
here’s your handle!”</em>. This side-effect don’t harm anyone, so we shouldn’t worry about it too much.</p>
<p>However, there are nasty side-effects, like binding resources to the OpenGL context.</p>
<p>So what are advantages of IO-bound designs? Well:</p>
<ul>
<li>simple: indeed, you have handles and side-effects and you have a (too) fine control of the
instruction flow;</li>
<li>performance, because <code>IO</code> is the naked real-world monad;</li>
<li>because <code>IO</code> is the high-order kinded type of any application (think of the <code>main</code> function),
an <code>IO</code> API is simple to use in any kind of application;</li>
<li>we can use <code>(MonadIO m) =&gt; m</code> to add extra flexibility and create interesting constraints.</li>
</ul>
<p>And drawbacks:</p>
<ul>
<li><code>IO</code> is very opaque and is not referentially transparent;</li>
<li><code>IO</code> is a dangerous type in which no one has no warranty about what’s going on;</li>
<li>one can fuck up everything if they aren’t careful;</li>
<li>safety is not enforced as in pure code.</li>
</ul>
<h2>What about luminance’s design?</h2>
<p>Since the beginning, luminance has been an API built to be simple, <em>type-safe</em> and <em>stateless</em>.</p>
<p><em>Type-safe</em> means that all objects you use belong to different type sets and cannot be mixed between
each other implicitely – you have to use explicit functions to do so, and it has to be meaningful.
For instance, you cannot create a buffer and state that the returned handle is a texture: the type
system forbids it, while in OpenGL, almost all objects are in the <code>GLuint</code> set. It’s very
confusing and you might end up passing a texture (<code>GLuint</code>) to a function expecting a framebuffer
(<code>GLuint</code>). Pretty bad right?</p>
<p><em>Stateless</em> means that luminance has no state. You don’t have a huge context you have to bind stuff
against to make it work. Everything is stored in the objects you use directly and specific context
operations are translated into a different workflow so that performance are not destroyed – for
instance luminance uses batch rendering so that it performs smart resource bindings.</p>
<p>Lately, I’ve been thinking of all of that. Either turn the API pure or leave it the way it is. I
started to implement a pure API using <em>self-recursion</em>. The idea is actually simple. Imagine this
<code>GPU</code> type and the <code>once</code> function:</p>
<pre><code class="language-haskell">import Control.Arrow ( (***), (&amp;&amp;&amp;) )
import Data.Function ( fix )

newtype GPU f a = GPU { runGPU :: f (a,GPU f a) }

instance (Functor f) =&gt; Functor (GPU f) where
  fmap f = GPU . fmap (f *** fmap f) . runGPU

instance (Applicative f) =&gt; Applicative (GPU f) where
  pure x = fix $ \g -&gt; GPU $ pure (x,g)
  f &lt;*&gt; a = GPU $ (\(f',fn) (a',an) -&gt; (f' a',fn &lt;*&gt; an)) &lt;$&gt; runGPU f &lt;*&gt; runGPU a

instance (Monad m) =&gt; Monad (GPU m) where
  return = pure
  x &gt;&gt;= f = GPU $ runGPU x &gt;&gt;= runGPU . f . fst

once :: (Applicative f) =&gt; f a -&gt; GPU f a
once = GPU . fmap (id &amp;&amp;&amp; pure)
</code></pre>
<p>We can then build pure values that will have a side-effect for resource acquisition and then hold
the same value for ever with the <code>once</code> function:</p>
<pre><code class="language-haskell">let buffer = once createBufferIO

-- later, in IO
(_,buffer2) &lt;- runGPU buffer
</code></pre>
<p>Above, the type of <code>buffer</code> and <code>buffer2</code> is <code>GPU IO Buffer</code>. The first call <code>runGPU buffer</code> will
execute the <code>once</code> function, calling the <code>createBufferIO</code> IO function and will return <code>buffer2</code>,
which just stores a pure <code>Buffer</code>.</p>
<p>Self-recursion is great to implement local states like that and I advise having a look at the
<code>Auto</code> type. You can also read <a href="http://phaazon.blogspot.fr/2015/03/getting-into-netwire.html">my article on netwire</a>, which uses self-recursion a lot.</p>
<p>However, I kinda think that a library should have well defined responsibilities, and building such
a pure interface is not the responsibility of luminance because we can have type-safety and a
stateless API without wrapping everything in that <code>GPU</code> type. I think that if we want such a pure
type, we should add it later on, in a 3D engine or a dedicated framework – and that’s actually what
I do for demoscene purposes in another, ultra secret project. ;)</p>
<p>The cool thing with luminance using <code>MonadIO</code> is the fact that it’s very easy to use in any kind
of type that developpers want to use in their applications. I really don’t like frameworks which
purpose is clearly not flow control that actually enforce flow control and wrapping types! I don’t
want to end up with a <code>Luminance</code> type or <code>LuminanceApplication</code> type. It should be simple to use
and seamless.</p>
<p>I actually start to think that I did too much about that pure API design idea. The most important
part of luminance should be type-safety and statelessness. If one wants a pure API, then they
should use FRP frameworks or write their own stuff – with free monads for instance, and it’s
actually funny to build!</p>
<p>The next big steps for luminance will be to clean the uniform interfaces which is a bit inconsistent
and unfriendly to use with render commands. I’ll let you know.</p>


title: Even more hindsight on Vim, Helix and Kakoune
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/more-hindsight-vim-helix-kakoune
feedurl: https://phaazon.net/blog/feed
guid:
https://strongly-typed-thoughts.net/blog/more-hindsight-vim-helix-kakouneWed, 24 May 2023 11:50:00 GMT

content:
<p>Oh my… 2023 has been such a trek so far… If you have missed my <a href="https://phaazon.net/blog/editors-in-2022">previous article</a>
about my thoughts about editors and development platforms, I think it’s probably the moment to have a look at it.</p>
<p>Today is the end of May 2023. <a href="https://helix-editor.com/">Helix</a> has been my primary editor for months now. I haven’t
come back to <a href="https://neovim.io/">Neovim</a>. In the previous article, I mentioned that I couldn’t give a fair opinion
about Helix because I had just started using it. Today, I think I have enough experience and usage (5 / 6 months) to
share what I think about Helix, and go a little bit further, especially regarding software development in general and,
of course, editing software.</p>
<p>However, before starting, I think I need to make a clear disclaimer. If you use Vim, Neovim, Emacs, VS Code, Sublime
Text, whatever, and you think that <em>“Everyone should just use whatever they want”</em>, then we agree and <strong>this is not
the point of this blog article</strong>. The goal is to discuss a little bit more than just speaking out obvious takes, but
please do not start waving off the topic because you think <em>“Everyone should use what they want”</em>. There is a place
for constructive debate even there. If you start arguing, then it means you want to debate, and then you need to be
ready to have someone with different arguments that will not necessarily go your way, nor your favorite editor.</p>
<p>Finally, if you think I haven’t used Vim / Neovim enough, just keep in mind that I have been using (notice the tense)
Vim (and by extension, Neovim) since I was 15, and I’m 31, so 16 years.</p>
<p>I have wrote that blog article already three times before deleting everything and starting over. I think I will make
another blog article about how I think about software, but here I want to stay focused on one topic: editors and
development environments.</p>
<h1>Helix</h1>
<p>From a vimmer perspective, Helix is weird. It reverses the verb and the motion (you don’t type <code>di(</code> to delete inside
parenthesis, but you type <code>mi(d</code> to first match the parenthesis and then delete). Then, you have the multi-selections
as a default way of doing things; Helix doesn’t really have the concept of a <em>cursor</em>, which is a design it gets from
<a href="https://kakoune.org/">Kakoune</a>, and I’ll explain what it means and implies later.</p>
<p>Some vimmers publicly talked about it. ThePrimeagen, for instance, made a Youtube video about it where he basically
just scratched the surface of the editor and <em>“Hell it’s not Vim so it’s not really good”</em>. He moved the video to
private then but I’m sure you can just look for Tweets. Many Vim afficionados react that way, which is not a very
serious way of trying out something new, especially if you happen to publicly talk about it to thousands of people. I
think it’s not fair to both the tool (here, Helix) and the people reading you, unless you are one of those Vim zealots
thinking you know everything better than everyone else and dismissing people’ points of views just because they are not
the same as yours.</p>
<p>Anyway, that reputation didn’t hold me from trying out, and the way I try software is simple: just give in and accept
to drop your habits and productivity. Of course, at work, I was still using Vim / Neovim, but switched to Helix on my
spare-time projects.</p>
<p>There are many aspects to talk about with Helix. The first one is that, contrary to what people who haven’t really and
seriously tried it, the difference with Vim is not only “reversed motion/verb and multi-cursors.” The first difference
is the <em>design</em> and the <em>direction</em>.</p>
<h2>Design and direction</h2>
<p>Helix is an editor that natively integrates many features that are most of the time plugins in others editors (even
in VS Code). The best examples here are <a href="https://tree-sitter.github.io/tree-sitter/">tree-sitter</a>, surrounding pairs (
being able to automatically surround regions of text with <code>(</code>, <code>[</code>, HTML tags, etc.), LSP UIs, fuzzy pickers, etc. This
is a <em>massive</em> and important difference because of two main things:</p>
<ol>
<li>Code maintenance.</li>
<li>User experience.</li>
</ol>
<p>About code maintenance, having all of those features natively integrated in the editor means that you are 100% sure that
if you get the editor to start, the features will work, and the editor developers will keep that updated within the
next iteration of the editor. For instance, having tree-sitter natively implemented (Rust) in Helix means that the
editor itself knows about tree-sitter and its grammars, highlights queries and features. The same thing goes for
surrounding-pairs or auto-pairs, for instance. If the team decides to change the way text is handled in buffers, then
the code for auto-pairs / surrounding-pairs <strong>will have to be updated for the editor to be releasable</strong>.</p>
<p>The user experience will then be better, because you get those nice features without having to start looking around
for a plugin doing it, with the problem of chosing the right one among a set of competing plugins. Plus the risk of
having your plugin break because it’s not written by the Helix team. Just install the software and start using those
features.</p>
<p>For now, Helix ships with a bunch of powerful features that makes it usable in a modern environment:</p>
<ul>
<li>tree-sitter support for semantic highligthing, semantic selection and navigation, etc. That also includes a native
support for getting grammars / queries from the CLI with <code>hx --grammar</code>.</li>
<li>LSP support, both in terms of features (go-to, references, implementations, incoming / outgoing calls, diagnostics,
inlay hints, etc. etc.) and in terms of UI. It’s the same UI for everyone.</li>
<li>DAP support for debugging (experimental at the time of writing).</li>
<li>Surrounding pairs.</li>
<li>Auto pairs.</li>
<li>Git integration (gutter diff on the right side of your buffer) and Git semantic object (go to next change, etc.).</li>
<li>Registers and user-defined registers (like in Vim), along with macros (but you won’t need them, trust me).</li>
<li>Native discoverability, including a command palette with <em>every possible available commands</em>, tagged with their
keybindings.</li>
<li>Many bundled themes (yes, <code>catppuccin</code> themes are there!).</li>
<li>Snappier than anything else.</li>
<li>Various integration with your system, including clipboards, external commands, etc.</li>
</ul>
<p>There is also one (major) thing I want to talk about and that deserves its own section: configuration.</p>
<h2>Configuration done right</h2>
<p>Helix treats configuration <em>the way it should be</em>: as data. Configuration in Helix is done in TOML. There is no
scripting implied, it’s only a rich object laid out as TOML sections. And this is a <em>joy</em> to use. For instance, this is
my current Helix configuration (excluding keys remapping, because my keyboard layout is bépo):</p>
<pre><code class="language-toml">theme = &quot;catppuccin_macchiato&quot;

[editor]
scroll-lines = 1
cursorline = true
auto-save = false
completion-trigger-len = 1
true-color = true
color-modes = true
auto-pairs = true
rulers = [120]
idle-timeout = 50

[editor.cursor-shape]
insert = &quot;bar&quot;
normal = &quot;block&quot;
select = &quot;underline&quot;

[editor.indent-guides]
render = true
character = &quot;▏&quot;

[editor.lsp]
display-messages = true
display-inlay-hints = true

[editor.statusline]
left = [&quot;mode&quot;, &quot;spinner&quot;, &quot;file-name&quot;, &quot;file-type&quot;, &quot;total-line-numbers&quot;, &quot;file-encoding&quot;]
center = []
right = [&quot;selections&quot;, &quot;primary-selection-length&quot;, &quot;position&quot;, &quot;position-percentage&quot;, &quot;spacer&quot;, &quot;diagnostics&quot;, &quot;workspace-diagnostics&quot;, &quot;version-control&quot;]
</code></pre>
<p>Notice the tree-sitter and LSP configuration. Yes. <em>None</em>.</p>
<p>This is so important to me. Because configuration is data, it is simple for Helix to expose it and present it to the 
user by reading the TOML file without caring about having any side-effects. Helix has a command line (<code>:</code>) where you can
tweak those options dynamically. And you can dynamically reload the configuration as well.</p>
<h2>Multi-selection centric</h2>
<p>The major difference, even before the reversed motion/verb thing, is the fact that Helix <em>doesn’t really</em> have a cursor.
It has the concept of <em>selections</em>. A selection is made of two entities:</p>
<ul>
<li>An <em>anchor</em>.</li>
<li>A <em>cursor</em>.</li>
</ul>
<p>The cursor is the part of the selection that moves when you extend the selection. The anchor, as the name implies, is
the other part that stays where it is: it’s anchored. <strong>By default, you have only one selection</strong> and the anchor is 
located at the same place as the cursor. It looks similar to any other editor. Things start to change when you begin
typing normal commands. Typing <code>l</code>, for instance, will move both the anchor and cursor to the right, making them a
single  visual entity. However, if you type <code>w</code>, the cursor will move to the end of the word while the anchor will move
to its beginning, visually selecting the word. If you type <code>W</code>, the anchor won’t move and only the cursor will move,
extending the selection. If you press <code>B</code>, it will move the cursor back one word, shrinking the selection. You can press
<code>&lt;a-;&gt;</code> to flip the anchor and the selection, which is useful when you want to extend on the left or on the right.</p>
<p>This concept of selection is really powerful because everything else is based on it. Pressing <code>J</code> will move the cursor
down one line, leaving the anchor on the current line, extending selected lines. Once something is selected, you can
operate on it, with <code>d</code>, <code>c</code>, <code>y</code>, <code>r</code>, etc. For instance, <code>wd</code> will select the word and delete it. <code>Jc</code> will extend the
selection with the next line and start changing. Selections in Helix are not just visual helps: they represent what
normal editing operations will work on, which is a great design, because you can extend, shrink and reason about them in
a much more seamless and natural way.</p>
<p>But it’s just starting. Remember earlier when I say that <strong>by default, you have only one selection</strong>? Well, you can have
many, and this is where Helix starts to really shine to me. The first way to create many selections is to press <code>C</code>. <code>C</code>
will duplicate your current selection on the next line. If you have the anchor at the same position as the cursor,
pressing <code>C</code> will make it like you have another cursor on the next line below. For instance, consider this text:</p>
<pre><code class="language-text">I love su|shi.
But I also love pizza.
</code></pre>
<p>The <code>|</code> is our cursor (but also anchor). If you press <code>C</code>, you will see something like this:</p>
<pre><code class="language-text">I love su|shi.
But I als|o love pizza.
</code></pre>
<p>But as I had mentioned, <code>C</code> duplicates <em>selections</em>. Let’s say you started like this, <code>&lt;</code> being the anchor and <code>|</code> the
cursor:</p>
<pre><code class="language-text">I love &lt;sus|hi.
But I also love pizza.
</code></pre>
<p>Pressing <code>C</code> now will do this:</p>
<pre><code class="language-text">I love &lt;sus|hi.
But I a&lt;lso| love pizza.
</code></pre>
<p>Once you have many selections, everything you type as normal commands will be applied to every selections. If I type
<code>f.</code>, it will set the anchor to the current cursor and extend the cursor to the next <code>.</code>, resulting in this:</p>
<pre><code class="language-text">I love sus&lt;hi|.
But I also&lt; love pizza|.
</code></pre>
<p>Press <code>&lt;a-;&gt;</code> to swap anchors and cursors:</p>
<pre><code class="language-text">I love sus|hi&lt;.
But I also| love pizza&lt;.
</code></pre>
<p>And then pressing <code>B</code> will do this:</p>
<pre><code class="language-text">I love |sushi&lt;.
But I |also love pizza&lt;.
</code></pre>
<blockquote>
<p>Erratum: <code>B</code> is not defined to this behavior in a vanilla Helix. I have remapped it to the Kakoune behavior. It
doesn’t change much to what I’m saying here, though.</p>
</blockquote>
<p>Multi-cursor is then not only a nice visual help, but also a completely new way of editing your buffers. Once you get
the hang of it, you don’t really think in terms of a single cursor but many selections, ranges, however you like to call
them.</p>
<h2>Getting more cursors</h2>
<p><code>C</code> is great, but this not something we usually use. Instead, we use features that don’t exist in Vim. I’m not entirely
sure how to call those, but I like to call them <em>selection generators</em>. They come in many different flavors, so I’ll
start with the easiest one and will finish with the most interesting and (maybe a bit?) obscure at first.</p>
<h3>Matching matching matching!</h3>
<p>The <code>m</code> key is Helix is a wonderful key. It’s the <em>match</em> key. It expects a motion and will change all your selections
to match the motion. For instance, <code>mia</code> <em>“matches inside arguments”</em> (tree-sitter). Imagine this context:</p>
<pre><code class="language-rust">fn foo(x: i32, y: |i32) {}

fn bar(a: String, |b: bool) {}
</code></pre>
<p>Press <code>mia</code> to get this:</p>
<pre><code class="language-rust">fn foo(x: i32, &lt;y: i32|) {}

fn bar(a: String, &lt;b: bool|) {}
</code></pre>
<p>Then, for instance, press <code>&lt;a-;&gt;</code> to flip the anchor and the cursor:</p>
<pre><code class="language-rust">fn foo(x: i32, |y: i32&lt;) {}

fn bar(a: String, |b: bool&lt;) {}
</code></pre>
<p><code>F,</code> to select the previous <code>,</code>:</p>
<pre><code class="language-rust">fn foo(x: i32|, y: i32&lt;) {}

fn bar(a: String|, b: bool&lt;) {}
</code></pre>
<blockquote>
<p>Erratum: same thing as with <code>B</code>; I have remapped it in my config. The default <code>B</code> doesn’t extend like this.</p>
</blockquote>
<p>And just press <code>d</code>:</p>
<pre><code class="language-rust">fn foo(x: i32) {}

fn bar(a: String) {}
</code></pre>
<p>It’s so logical, easy to think about and natural. Someting interesting to notice, too, is that contrary to Vim, which
has many keys doing mainly the same thing, making things weird and not really well designed. For instance, <code>vd</code> selects
the current character and delete it. <code>vc</code> deletes the current character and puts you into insert mode. <code>s</code> does exactly 
the same. Why would you have a key in direct access doing something so specific? If you want to delete a word, you 
either press <code>vwd</code>, or more simply in Vim, <code>dw</code>. All of that is already confusing, but it doesn’t end there. <code>x</code> deletes
the current character, but <code>x</code> is actually <em>cut</em>, so if you select a line with <code>V</code> and press <code>x</code>, it will cut the line.
Press <code>Vc</code> to change a line… or just <code>S</code>. What?</p>
<p>All those shortcuts feel like exceptions you have to learn, and it’s a good example of a flawed design. On the other
side, Helix (which is actually a Kakoune design it’s based on), have a single character to delete something: <code>d</code>. Since
the editor has multi-selections as a native editing entity, all of those situations will imply using the <code>d</code> key:</p>
<ul>
<li>Deleting the current character: <code>d</code>.</li>
<li>Deleting the next word: <code>wd</code>.</li>
<li>Deleting the current line: <code>xd</code> (<code>x</code> selects and extend the line).</li>
<li>Deleting delimiters but not the content: <code>md(</code>.</li>
<li>Etc. etc.</li>
</ul>
<p>That applies to everything.</p>
<h3>Selecting, splitting, keeping and removing</h3>
<p>The design of editing in Helix (Kakoune) is to be interactive and iterative. For instance, consider the following:</p>
<pre><code class="language-rust">  pub fn new(
    line_start: usize,
    col_start: usize,
    line_end: usize,
    col_end: usize,
    face: impl Into&lt;String&gt;,
  ) -&gt; Self {
    Self {
      line_start,
      col_start,
      line_end,|
      col_end,
      face: face.into(),
    }
  }
</code></pre>
<p>Let’s say we would like, to begin with, select very quickly every arguments type and switch them to <code>i32</code>. Many ways
of doing that, but let’s see one introducing a great concept: <em>selecting</em>. Selecting allows you to create new 
selections that satisfy a regex. The default keybinding for that is <code>s</code> for… select (woah). <code>s</code> always applies to the 
current Selections (notice the use of plural, it will be useful later). We then need to start selecting something. Here,
we can just press <code>mip</code> to select inside the paragraph, since our cursor is right after <code>line_end</code>:</p>
<pre><code class="language-rust">&lt; pub fn new(
    line_start: usize,
    col_start: usize,
    line_end: usize,
    col_end: usize,
    face: impl Into&lt;String&gt;,
  ) -&gt; Self {
    Self {
      line_start,
      col_start,
      line_end,
      col_end,
      face: face.into(),
    }
  }|
</code></pre>
<p>We have the whole thing selected. Press <code>s</code> to start selecting with a regex. We want the arguments, so let’s select
everything with <code>:</code> and press <code>s:</code> and return:</p>
<pre><code class="language-rust">  pub fn new(
    line_start&lt;:| usize,
    col_start&lt;:| usize,
    line_end&lt;:| usize,
    col_end&lt;:| usize,
    face&lt;:| impl Into&lt;String&gt;,
  ) -&gt; Self {
    Self {
      line_start,
      col_start,
      line_end,
      col_end,
      face&lt;:| face.into(),
    }
  } 
</code></pre>
<p>See how it created a bunch of selections for us. Also, notice that it selected <code>face: face.into()</code>, which is not
correct. We want to <em>remove</em> that selection. Again, several ways of doing it. Something to know is that, Helix (Kakoune)
has the concept of <em>primary</em> selection. This is basically the selection on which you are going to apply actions first,
like LSP hover, etc (it would be a madness to have LSP hover applies to all selections otherwise!). You can cycle the
primary selection with <code>(</code> and <code>)</code>. Once you reach the one you want, you can press <code>&lt;a-,&gt;</code> to just drop the selection.
However, we don’t want to cycle things. We want a faster way.</p>
<p>Let’s talk about <em>removing</em> selections. The default keybinding is <code>&lt;A-K&gt;</code>. However, here, our selections are all
about the same content (the <code>:</code>). As mentioned before, pressing <code>x</code> will select the current line of every selections:</p>
<pre><code class="language-rust">  pub fn new(
&lt;   line_start: usize,|
&lt;   col_start: usize,|
&lt;   line_end: usize,|
&lt;   col_end: usize,|
&lt;   face: impl Into&lt;String&gt;,|
  ) -&gt; Self {
    Self {
      line_start,
      col_start,
      line_end,
      col_end,
&lt;     face: face.into(),|
    }
  } 
</code></pre>
<p>Let’s filter selection and remove the one matching a pattern. In our case, let’s remove selections with a
<code>(</code> in them: <code>&lt;A-K&gt;\(</code> and return:</p>
<pre><code class="language-rust">  pub fn new(
&lt;   line_start: usize,|
&lt;   col_start: usize,|
&lt;   line_end: usize,|
&lt;   col_end: usize,|
&lt;   face: impl Into&lt;String&gt;,|
  ) -&gt; Self {
    Self {
      line_start,
      col_start,
      line_end,
      col_end,
      face: face.into(),
    }
  } 
</code></pre>
<p>Now press <code>_</code> to shrink the selections to trim leading and trailing whitespaces:</p>
<pre><code class="language-rust">  pub fn new(
    &lt;line_start: usize,|
    &lt;col_start: usize,|
    &lt;line_end: usize,|
    &lt;col_end: usize,|
    &lt;face: impl Into&lt;String&gt;,|
  ) -&gt; Self {
    Self {
      line_start,
      col_start,
      line_end,
      col_end,
      face: face.into(),
    }
  } 
</code></pre>
<p>Imagine that we have changed our mind and now we actually want to change the <code>usize</code> to <code>i32</code>. We can use the <em>keep</em>
operator, which is bound to <code>K</code> by default. Press <code>Kusize</code> and return to get this:</p>
<pre><code class="language-rust">  pub fn new(
    &lt;line_start: usize,|
    &lt;col_start: usize,|
    &lt;line_end: usize,|
    &lt;col_end: usize,|
    face: impl Into&lt;String&gt;,
  ) -&gt; Self {
    Self {
      line_start,
      col_start,
      line_end,
      col_end,
      face: face.into(),
    }
  } 
</code></pre>
<p>Another possible way is to press <code>susize</code> to only select the <code>usize</code> directly, which might be wanted if you want to 
change them quickly to <code>i32</code>, for instance.</p>
<p>The last operation that I want to mention is <em>splitting</em>. It’s introduced with <code>S</code> and will spawn several cursors
separated by a regex. For instance, consider:</p>
<pre><code class="language-rust">let |array = [1.0, 20.32, 3., 4.35];
</code></pre>
<p>Let’s say you’d like to select the numbers. With the methods described above, it’s probably challenging. With the
splitting command, it’s much easier. Put your cursor anywhere in the list and press <code>mi[</code> to select inside of it:</p>
<pre><code class="language-rust">let array = [&lt;1.0, 20.32, 3., 4.35|];
</code></pre>
<p>Then simply press <code>S,</code> to split the selection into selections separated by commas. You should end up with this:</p>
<pre><code class="language-rust">let array = [|1.0&gt;,| 20.32&gt;,| 3.&gt;,| 4.35&gt;];
</code></pre>
<p>Pressing <code>_</code> will shrink the selections to remove leading and trailing spaces:</p>
<pre><code class="language-rust">let array = [|1.0&gt;, |20.32&gt;, |3.&gt;, |4.35&gt;];
</code></pre>
<p>And here you have it! Now remember that you can combine all of this methods with semantic objects, like <code>mif</code> for
<em>inside functions</em>, <code>mat</code> for <em>around type</em>, <code>mia</code> for <em>inside arguments</em>, and many more. Recall that you can do that
<em>on each selection</em>, allowing for really powerful workflows.</p>
<h3>Do I really use all that at work / spare time projects?</h3>
<p><strong>Hell yes.</strong> It’s a muscular memory thing. For instance, I oftentimes the need to not only replace occurrences of 
patterns, like <code>fooN</code> — with <code>N</code> being a number — into <code>logger.fooN</code>, <strong>but I often need to change the structure around
those occurrences.</strong> And here, Helix really stands out. In Vim, you’d have to use a super ugly regex, completely blind,
and eventually a macro. The interactive and iterative approach of Helix is so much more powerful to me. For instance,
for the case described above: <code>%</code> to select the whole buffer, <code>sfoo.</code> to select <code>foo</code> with a single character
afterwards, then return, <code>clogger.foo</code> to replace with <code>logger.foo</code>, and still in insert mode, <code>&lt;C-r&gt;&quot;</code> to paste what
was yanked by the <code>c</code> operation. Here, the default register, <code>&quot;</code>, makes a <em>lot</em> of sense, because this register is local
to each selection, making this replace operation trivial and interactive.</p>
<p>Another example is something like this:</p>
<pre><code class="language-rust">const COLORS: [Color; 3] = [
  Color {
    r: 255,
    g: 0,
    b: 0,
  },
  Color {
    r: 0,
    g: 255,
    b: 0,
  },
  Color {
    r: 0,
    g: 0,
    b: 255,
  },
];
</code></pre>
<p>Imagine that you want to change every <code>Color</code> constructor to a function call, that does something like
<code>Color::rgb(r, g, b)</code>. Doing that interactively and iteratively in Helix is so easy. I’d put my cursor anywhere in that
block, press <code>mi[</code> to select everything inside <code>[]</code>, then <code>sColor&lt;cr&gt;</code> to create three cursors on the <code>Cursor</code>, and from
that moment, it’s just ping-pong-ing between normal mode and insert mode like you would do with a single selection. You
will be using <code>f</code>, <code>t</code>, <code>miw</code> etc. select things but the idea is the same, and the three occurrences will be updated at
once.</p>
<h2>A simpler editor overall</h2>
<p>Contrary to other famous editors and IDEs, Helix is not supposed to be extendable; it doesn’t try to solve more
problems than it should (and you will see in the Kakoune section that we can even push that to another extreme).
Something like Neovim is a bit of a disguised IDE. Yes, a vanilla Neovim with no plugins and no configuration is just
a very basic (and I would dare say <em>featureless</em>) editor. It won’t have LSP working. It won’t have tree-sitter working
either. Nothing for git integrated. Nothing for delimiters, nothing for pickers, nothing for any modern development. The
power of something like Emacs, Neovim etc. is to build on <em>extensibility</em>.</p>
<p>I used to enjoy that, until I came to the realization that, <em>perhaps</em>, it would be great to put things into perspective: 
is extensibility something we actually want? What do we try to solve with it? Well, we extend tools to add new features 
and new behaviors. We extend things so that the native / core tool ships with minimal features but doesn’t prevent 
people from adding specific and customized capabilities.</p>
<h3>Extensibility</h3>
<p>But is extensibility the only way to achieve that goal? The thing with extensibility is that:</p>
<ol>
<li>You have to build in advance for it. You cannot ship an editor without any extensibility support and expect it to
be an emergent feature. You have to add a basic support for it, whether it’s a plugin system, a
dynamic / relocatable system (<code>.so</code> / <code>.dylib</code> / <code>.dll</code>), a scripting language, a JIT, etc.</li>
<li>Extensibility is always walled.</li>
</ol>
<p>The last point is important. Extending a software <strong>requires the environment to adapt to the specifities of what you’re
extending</strong>, and that will require the environment to know about the specifities. Here, the environment could be
anything <strong>outside of Neovim</strong>. What it means is that, you’re not going to use external tools, but you are going to use
the interfaces, scripting languages, DSLs etc. of the tool you want to extend.</p>
<p>For instance, people might argue that extending Neovim is great because it only requires learning Lua, which is not
specific to Neovim, but that it is not actually true nor acurate. You have to learn <em>“Neovim Lua”</em>, which is basically 
its own beast. It’s like the standard library, but for Neovim. It will provide you with APIs you can use — and only 
use — to add new features to Neovim.</p>
<p>The same argument can be made to <em>any extensible</em> editor. VS Code, Emacs, etc.</p>
<h3>Composability</h3>
<p>Another way to add features and behaviors is to add them <em>externally</em>, by leveraging the tools themselves and compose
them instead of pushing more features into them. That vision is not very popular and famous and I’m not entirely sure
why. For instance, Vim has <a href="https://github.com/tpope/vim-fugitive">vim-fugitive</a>, a Git client for Vim / Neovim. It has
2k commits, between 8k-9k lines of code… and can be used only in Vim and Neovim. Yes, it extends and adds features
<em>inside</em> those editors, but still. If at some point you decide to switch to another editor, you can forget about this
plugin. This is even worse with something like <a href="https://magit.vc/">magit</a>, which is the best Git client I haver ever
used. Yet I don’t use it anymore, because it’s an Emacs plugin. What a shame.</p>
<p>Now repeat that reasoning for all the plugins you use. That has led some people to just install as few plugins as
possible and switch to composability.</p>
<p>Composability is the same concept as in code. You have two systems <code>A</code> and <code>B</code> doing <code>thingA</code> and <code>thingB</code>, and you 
write some <em>piping</em> glue code / script to connect both. People use many different languages to glue things together. 
Among the most famous approaches:</p>
<ul>
<li>Python.</li>
<li>Perl, especially when dealing with filtering text.</li>
<li>The shell, and especially any POSIX.2 compliant shell (no, don’t start talking about <code>fish</code>).</li>
</ul>
<p>I have shell functions that I source when I start my shell; for instance, one called <code>p</code>, to switch to my spare-time
projects:</p>
<pre><code class="language-sh">p () {
  proj_dir=${PROJ_DIR:-~/dev}
  project=$(ls $proj_dir | sk --prompt &quot;Switch to project: &quot;)
  [ -n &quot;$project&quot; ] &amp;&amp; cd $proj_dir/$project
}
</code></pre>
<p>This is a great example of composability, which composes the content of a project directory (the <code>$PROJ_DIR</code> environment
variable, or <code>~/dev</code> by default), with the <code>sk</code> fuzzy finder, and then changes the directory to whatever the user has
picked. I use that script all the time to quickly move to my various projects.</p>
<p>Notice that, <code>sk</code>, <code>fzf</code>, etc. already are tools that implement fuzzy searching for arbitrary inputs. Tools such as
<code>find</code> or <code>fd</code>, who are so far in my experience the fastest programs to look for stuff, can be composed with shell
pipes as well and integrated into your work environment.</p>
<p>Then the question starts to appear: why do editors / plugins re-implement all of that to have it inside the editor? It’s
pretty apparent in the Neovim community, but it often ends up with abandonware plugins, and harder to maintain
editors.</p>
<p>While I was wondering about all that, I was getting pretty productive with Helix, enjoying its simpler design, data
configuration, better editing features and overall way more stable experience. I remembered that most of the Helix
design came from Kakoune. And I started to think about one (not so) crazy idea: should I have a look at Kakoune?</p>
<p>And let’s enter Kakoune.</p>
<h1>Kakoune</h1>
<p>As mentioned above, Helix is <em>heavily</em> inspired by Kakoune. The main difference, from the surface, with distance, is
that Helix comes with more bundled features, like LSP, tree-sitter, pickers, etc. However, there are more (drastic)
differences that I need to talk about.</p>
<p>The first thing is, again, the design. And for that, I really need to quote a part from the
<a href="https://github.com/mawww/kakoune/blob/master/doc/design.asciidoc">excellent design doc</a> written by
<a href="https://github.com/mawww">@mawww</a>. The part that is the most interesting to me is, obviously, <em>composability</em>.</p>
<blockquote>
<p>Being limited in scope to code editing should not isolate Kakoune from its environment. On the contrary, Kakoune is
expected to run on a Unix-like system alongside a lot of text-based tools, and should make it easy to interact with
these tools.</p>
<p>For example, sorting lines should be done using the Unix sort command, not with an internal implementation. Kakoune
should make it easy to do that, hence the <code>|</code> command for piping selected text through a filter.
The modern Unix environment is not limited to text filters. Most people use a graphical interface nowadays, and
Kakoune should be able to take advantage of that without hindering text mode support. For example, Kakoune enables
multiple windows by supporting many clients on the same editing session, not by reimplementing tiling and tabbing.
Those responsibilities are left to the system window manager.</p>
</blockquote>
<p>And this is one of the most important things about Kakoune to me. First, it goes into the direction I’ve been wanting
for years (I’ll let you read my previous blog articles about editors and production environments; I’m basically saying
the same thing). And second, it ensures that the native editor remains small and true to its scope, ensuring an easier
maintenance, hence less bugs and more stable. Also, it doesn’t blindly ignore what everyone else is doing.</p>
<p>Let’s start with an example. Yes, Kakoune doesn’t have a fuzzy picker to pick your files. However, as mentioned above,
it composes well with its environment. It does that via different mechanisms (shell blocks, FIFOs, UNIX sockets, etc.).
Here, we can just use whatever we like to get a list of files, and let Kakoune ask the user which files to open. We then
simply use the selected value and open it. In order to do that, you need to read the design doc to understand a couple
of other things, such as the section about <em>interactive use and scripting</em>. Quoting:</p>
<blockquote>
<p>As an effect of both Orthogonality and Simplicity, normal mode is not a layer of keys bound to a text editing language
layer. Normal mode <strong>is</strong> the text editing language.</p>
</blockquote>
<p>Typing normal commands in a <code>.kak</code> file is then the way to go. And then, coming back to the fuzzy picker example, here
are three commands defined in my <code>kakrc</code>:</p>
<pre><code>## Some pickers
define-command -hidden open_buffer_picker %{
  prompt buffer: -menu -buffer-completion %{
    buffer %val{text}
  }
}

define-command -hidden open_file_picker %{
  prompt file: -menu -shell-script-candidates 'fd --type=file' %{
    edit -existing %val{text}
  }
}

define-command -hidden open_rg_picker %{
  prompt search: %{
    prompt refine: -menu -shell-script-candidates &quot;rg -in '%val{text}'&quot; %{
      eval &quot;edit -existing  %sh{(cut -d ' ' -f 1 | tr ':' ' ' ) &lt;&lt;&lt; $kak_text}&quot;
    }
  }
}
</code></pre>
<p>As you can see, it’s just about composing known and well written tools together. Another example? Alright. Kakoune
doesn’t have splits, but I still want them. Let’s go:</p>
<pre><code>## kitty integration
define-command -hidden kitty-split -params 1 -docstring 'split the current window according to the param (vsplit / hsplit)' %sh{
  kitty @ launch --no-response --location $1 kak -c $kak_session
}

## zellij integration
define-command -hidden zellij-split -params 1 -docstring 'split (down / right)' %sh{
  zellij action new-pane -cd $1 -- kak -c $kak_session
}

define-command -hidden zellij-move-pane -params 1 -docstring 'move to pane' %sh{
  zellij action move-focus $1
}

## tmux integration
define-command tmux-split -params 1 -docstring 'split (down / right)' %sh{
  tmux split-window $1 kak -c $kak_session
}

define-command tmux-select-pane -params 1 -docstring 'select pane' %sh{
  tmux select-pane $1
}
</code></pre>
<p>The design is not extensible: it’s composable, and all in all, it makes so much more sense to me.</p>
<h2>Kakoune vs. the rest</h2>
<p>If you are used to Helix, then Kakoune with a bit of configuration will feel very similar to Helix. Of course, you will
have to look around for LSP and tree-sitter support. The way we do that is by adding external processes to interact with
Kakoune servers / clients via UNIX sockets, FIFO pipes, etc.. Kakoune doesn’t know anything about LSP or tree-sitter,
but you can write a binary in any language you want and send remote commands to control the behavior of Kakoune.</p>
<ul>
<li>For LSP: <a href="https://github.com/kak-lsp/kak-lsp">kak-lsp/kak-lsp</a></li>
<li>For tree-sitter: <a href="https://github.com/phaazon/kak-tree-sitter">phaazon/kak-tree-sitter</a> (I’m stil working on it and it
lacks documentation but is usable).</li>
</ul>
<p>The interesting aspect with those tools is that, in <em>theory</em>, we could adapt them to make them editor independent. If
more editors adopted the strategy of Kakoune (composing via the shell), we wouldn’t even have to write other binaries to
add LSP / tree-sitter support, which is an interesting aspect.</p>
<p>I plan on writing a blog article detailing the design of <code>kak-tree-sitter</code>, because I think it’s a good source of
knowledge regarding UNIX and tree-sitter.</p>
<p>Besides that, Kakoune is way more mature than Helix, in the sense that it has some specificities to some edge cases with
multi-selection features (such as, what happens when you have multiple cursors inside text looking like function
argument lists, and you type <code>mia</code> to select them, but some selections are actually not arguments? Kakoune will remove
the mismatched selections, which is what we would expect, while Helix…… erm it’s complicated! but currently, it will
keep the selections around, which is confusing and dangerous).</p>
<p>Kakoune has a <em>wonderful</em> feature called <strong>marks</strong>. Marks are different from what you have in Vim. They use a specific
register to record the current selections and eventually restore them later, supporting merging selections and editing
commands. An example that I love doing; imagine the following snippet:</p>
<pre><code class="language-markdown">Thank you to [@NAME](https://github.com/NAME) for their contribution.
</code></pre>
<p>Let’s say you want to have a cursor on each <code>NAME</code>. Easy, with <code>s</code>. You just select the whole thing (you can select the
whole line with <code>x</code> for instance), then <code>sNAME</code>, return, then <code>c</code> to start changing with whatever you want.</p>
<p>Now, imagine this instead:</p>
<pre><code class="language-markdown">Thank you to [@](https://github.com/) for their contribution.
</code></pre>
<p>How do you insert at the same time at the right of <code>@</code> and before the <code>)</code>? Getting two selections will be hard (or you
will end up writing crazy regexes; remember, we are not using Vim anymore!). A super easy way to do it is to move your
selection to <code>@</code>:</p>
<pre><code class="language-markdown">Thank you to [@|](https://github.com/) for their contribution.
</code></pre>
<p>Press <code>Z</code> to register the current selection (you have only one). Then move the cursor to the <code>)</code>:</p>
<pre><code class="language-markdown">Thank you to [@](https://github.com/|) for their contribution.
</code></pre>
<p>And then press <code>&lt;a-z&gt;a</code> to merge the current selection to the one(s) already stored. You end up with this:</p>
<pre><code class="language-markdown">Thank you to [@|](https://github.com/|) for their contribution.
</code></pre>
<p>Two cursors at the right places! This is extremely powerful and a feature that should arrive in Helix, but not sure
exactly when.</p>
<p>A pretty other important thing to say about Kakoune is that it has a server/client design that allows to share session
contexts. That is the main mechanisms used to implement native splits (via Kitty, tmux, whatever), but also many other
features, such as project isolation, viewing the same buffers with different highlighters, etc. etc.</p>
<h2>…but it’s not perfect</h2>
<p>There is one important thing I need to mention. I have been playing with Kakoune for a while now, and I have been
working on <code>kak-tree-sitter</code> for almost as long as I’ve been working Kakoune (couple of months). And there is one issue
with the UNIX approach.</p>
<p>See, tree-sitter, LSP, DAP, git gutters, etc. All those things are pretty <em>fundamental</em> to a modern text editor.
Externalizing them (Kakoune) is an interesting take, but I’m not entirely sure Kakoune is still doing it completely
correctly.</p>
<p>The main problem is <em>social intelligence</em>. Because all tooling is now externalized, many people can come up with
their own efforts. For instance, <code>kak-lsp</code> and <code>kak-tree-sitter</code> are completely separate projects, and they should
remain that way (for many reasons; scopes, maintenance, dependencies, etc.). However, in order to operate on the editor
contents, both programs must interact with the editor. That implies:</p>
<ul>
<li>Retrieving buffer contents.</li>
<li>Performing some actions that can mutate (and overlap / hijack) each other.</li>
<li>Insert hooks that might be incompatible between them.</li>
</ul>
<p>This problem is important, because dumping the whole content of a big buffer to an external process is one thing, doing
it for every different external processes is a massive overhead. Because I have read a bit the source code of <code>kak-lsp</code>,
I know that we (<code>kak-tree-sitter</code>) are doing similar things: we do use FIFOs to write buffer content and communicate
with our servers / daemons without going through the disk. But we are doing it that <em>twice</em>. And it’s just two projects;
any dissociate projects needing to access the buffer content will probably perform similar things.</p>
<p>That is a massive problem to me and I’m not sure how I feel about it. I’m not against sending the content of a buffer
via a FIFO to an externalized program — I actually think it’s a pretty good design —, but doing it for every
integration… I’m not exactly sure what would be the best solution, but maybe something that would snapshot a buffer
inside some POSIX shared memory (with mutable lock access if needed) could be one way to go. Honestely, I am not sure.</p>
<p>All of that to say that, the take of Helix is pretty good here, because all of those UNIX problems are not there in that
editor: everything runs in the same process, inside the same memory region. I will come back to this problem with my
next article on <code>kak-tree-sitter</code> and its design.</p>
<h1>Conclusion</h1>
<p>Today, I’m mainly using both Kakoune and Helix. Helix still has some bugs, even with tree-sitter (which my daemon
doesn’t have, funnily!), so I sometimes use one or the other tool.</p>
<p>I have learned so many things lately, with both Helix and Kakoune (especially Kakoune, it made me love UNIX even more).
All of that echoes the disclaimer I made earlier: yes, Vim and Neovim are good, but Kakoune and Helix are so much better
to me. Better designs, better editing experiences, largely snappier, more confidence in the direction of the
native code (because a much, much smaller codebase). Helix is written in Rust, Kakoune in C++, but what matters is the
actual design. To me, Kakoune is by far the best designed software I have ever seen, and for that, I really admire the
work of <a href="https://github.com/mawww">@mawww</a> and everyone else involved in the project. My contribution to the Kakoune
world with <a href="https://github.com/phaazon/kak-tree-sitter">kak-tree-sitter</a> is, I hope, something that will help and drive
more people in. I will write another blog article about that, with the pros., cons. and tradeoffs. of composability in
editors.</p>
<p>In the meantime, have fun and use the editor you love, but remember to have a look around and stay open to change! Keep
the vibes!</p>
<blockquote>
<p>I would like to thank <a href="https://github.com/taupiqueur/">@Taupiqueur</a>, who played an important role into making me
undersand Kakoune and eventually fall in love with — the editor, I mean! :)</p>
</blockquote>


guid: https://strongly-typed-thoughts.net/blog/rank-n-rustSat, 19 May 2018 12:48:00 GMT
title: Rank-n functions in Rust?
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/rank-n-rust
feedurl: https://phaazon.net/blog/feed
content:
<p>Yesterday, I came across a very simple problem <a href="https://www.reddit.com/r/rust/comments/8kh94b/is_there_any_way_to_have_move_semantics_rankn">that I summed up here</a>.
The idea is the following:</p>
<ul>
<li>You want to implement a unary function <code>foo</code> that takes a function <code>F</code> as sole argument and
returns its result.</li>
<li><code>F</code> is a unary function that takes a function <code>Λ</code> as sole argument.</li>
<li><code>Λ</code> doesn’t take any argument and return a type <code>T</code>.</li>
<li>We want linear typing here, but because it would make the blog post more complicated, we’ll
ignore that property.</li>
</ul>
<p>In <strong>Haskell</strong>, you would represent this scenario like this:</p>
<pre><code>data T = T -- we’re not interested in this type yet

        --     v this is F
        --------------
foo :: ((() -&gt; T) -&gt; b) -&gt; b
         -------
         -- ^ this is Λ

-- or for people less familiar with Haskell, here’s a decomposed version (but still the same thing)
type Λ = () -&gt; T
type F b :: Λ -&gt; b
foo :: F b -&gt; b
</code></pre>
<blockquote>
<p>Note: it’s pretty much known in the Haskell world but maybe not in others: if you have a value,
like <code>a</code>, it’s isomorphic to a unary function that returns this value and takes <code>()</code> as single
argument – <code>() -&gt; a</code>.</p>
</blockquote>
<blockquote>
<p>Note²: for curious, in Haskell, the syntax <code>a -&gt;. b</code> is a linear arrow. More about this on
<a href="https://ghc.haskell.org/trac/ghc/wiki/LinearTypes">linear Haskell</a>. I didn’t include the syntax in the snippet because it would make the post too
complicated for our purpose.</p>
</blockquote>
<blockquote>
<p>Note³: this is a rank-2 type function, because it’s a function that takes a function (1) that
takes another function (2). There’re levels of “nesting” there.</p>
</blockquote>
<p>In <strong>Rust</strong>, argh… <strong>Rust</strong> is not as powerful as <strong>Haskell</strong> for complex abstractions – even though
we’re working on making it better and better. To me, <strong>Rust</strong> competes with <strong>Haskell</strong>, but the
current blog post will show you that it still has a lot to learn from its parent! :)</p>
<p>Let’s sketch something in <strong>Rust</strong>.</p>
<pre><code>struct T; // we’re not interested in this type yet

fn foo&lt;F, B&gt;(f: F) -&gt; B where F: FnOnce(???) -&gt; B {
  f(|| 42)
}
</code></pre>
<p>What should we put in place of the <code>???</code> placeholder? Clearly, looking at the <strong>Haskell</strong> snippet,
we want a function that produces a <code>T</code>. Something like that:</p>
<pre><code>fn lambda() -&gt; T
</code></pre>
<p>So we could use the type <code>fn () -&gt; T</code>, but that will not work, because a lambda has not such a
type – you can see the lambda <code>|| 42</code>. Let’s try something else.</p>
<pre><code>fn foo&lt;F, B&gt;(f: F) -&gt; B where F: FnOnce(impl FnOnce() -&gt; T) -&gt; B
</code></pre>
<p>If you look at the <a href="https://github.com/rust-lang/rfcs/blob/master/text/1522-conservative-impl-trait.md">impl Trait RFC</a>,
you see that it’s a <strong>Rust</strong> construct to build a contract that lets you handle a value through
<em>traits</em> without knowing the type – we call that an <em>existential</em>. The type is picked by the
<em>callee</em>, which in our case is the function itself.</p>
<p>However, this syntax doesn’t work because – as for now – <code>impl Trait</code> works only in return position
and the consensus seems to have agreed that when <code>impl Trait</code> will be implemented in argument
position, <strong>it will not mean an existential, but a universal</strong>, e.g.:</p>
<pre><code>fn bar&lt;I: Iterator&lt;Item = u32&gt;&gt;(i: I)
fn bar&lt;I&gt;(i: I) where I: Iterator&lt;Item = u32&gt;
fn bar(i: impl Iterator&lt;Item = u32&gt;)
</code></pre>
<p>Are all the same thing.</p>
<blockquote>
<p>Note: I’ve made a ranting about the decision to give universal semantics to <code>impl Trait</code> in
argument position <a href="https://github.com/rust-lang/rust/issues/34511#issuecomment-390358483">here</a>.
I don’t want to start a type theory war, if it’s the consensus, then we have to accept it and
move on. It’s just a bit a pity. Topic closed, let’s get back to our sheep!</p>
</blockquote>
<p>So we cannot use that. Argh! Let’s try something else then!</p>
<pre><code>fn foo&lt;F, G, B&gt;(f: F) -&gt; B where F: FnOnce(G) -&gt; B, G: FnOnce() -&gt; T
</code></pre>
<p>This cannot work because here you have two universals <code>F</code> and <code>G</code>, meaning that <code>G</code> will be picked
by the caller while you want to decide of its type directly in your function (remember the lambda).</p>
<pre><code>fn foo&lt;F, B&gt;(f: F) -&gt; B where F: for&lt;G: FnOnce() -&gt; T&gt; FnOnce(G) -&gt; B
</code></pre>
<p>I like this one <em>a lot</em>. It uses a concept called <a href="https://doc.rust-lang.org/nomicon/hrtb.html">HRTB</a> in <strong>Rust</strong> which you can see by the
<code>for&lt;_&gt;</code> syntax. However, this will not work because as for today, <a href="https://doc.rust-lang.org/nomicon/hrtb.html">HRTB</a> only works with lifetimes.</p>
<blockquote>
<p>Ok, that seems a bit desperate!</p>
</blockquote>
<p>Hm, how about this:</p>
<pre><code>fn foo&lt;F, B&gt;(f: F) -&gt; B where F: FnOnce(Box&lt;FnOnce() -&gt; T) -&gt; B
</code></pre>
<p>Even though that seems to typecheck, you won’t ever be able to use <code>Λ</code> here because you cannot move
out of a <code>Box</code> with <code>FnOnce</code>. This limitation is removed with the <code>FnBox</code> type:</p>
<pre><code>fn foo&lt;F, B&gt;(f: F) -&gt; B where F: FnOnce(FnBox() -&gt; T) -&gt; B
</code></pre>
<p>However:</p>
<ul>
<li><code>FnBox</code> is unstable and will only work on nightly right now.</li>
<li>It <em>boxes</em> the function, which is not what we want there.</li>
</ul>
<blockquote>
<p>Ok, that seems desperate. Period.</p>
</blockquote>
<p>Well, if we only stick to those tools (i.e. functions, type variables and trait bounds), currently,
yeah, there’s no solution. However, if we add the traits to our toys, we can work around the
problem:</p>
<pre><code>trait GConsumer {
  type Output;

  fn call_once&lt;G&gt;(self, f: G) -&gt; Self::Output where G: FnOnce() -&gt; i32;
}

fn foo&lt;F&gt;(f: F) -&gt; F::Output where F: GConsumer {
  f.call_once(|| 32) // hurray!
}
</code></pre>
<p>This snippet is actually quite interesting. We introduce the <em>existential</em> via a trait. The
existential is encoded with the <code>G</code> type variable of the <code>call_once</code> method of <code>GConsumer</code>. If you
look at that function from <code>foo</code> perspective, <code>G</code> is definitely an <em>existential</em>! – and it’s a
<em>universal</em> for <code>call_once</code>.</p>
<p>I really like this idea, because it’s simple and legacy <strong>Rust</strong>. But I feel a bit confused, because
it’s simple to define but hard to use. Instead of just passing a closure to <code>foo</code>, people will now
have to implement a separate type on which will be called <code>call_once</code>. That’s a lot of boilerplate.</p>
<p>I hope we’ll come to a better solution to this problem – if you can do it with any trait, it should
be possible with <code>FnOnce</code> and a few candies into the <strong>Rust</strong> language itself.</p>
<p>That’s all for me for today. Keep the vibes and have a nice weekend!</p>


guid: https://strongly-typed-thoughts.net/blog/luminance_rustFri, 29 Apr 2016 00:00:00 GMT
title: Porting a Haskell graphics framework to Rust (luminance)
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance_rust
feedurl: https://phaazon.net/blog/feed
content:
<p>I wanted to write that new article to discuss about something important I’ve been doing for several 
weeks. It’s actually been <em>a month</em> that I’ve been working on <em>luminance</em>, but not in the usual
way. Yeah, I’ve put my <em>Haskell</em> experience aside to… port <em>luminance</em> into <em>Rust</em>! There are
numerous reasons why I decided to jump in and I think it could be interesting for people to know
about the differences I’ve been facing while porting the graphics library.</p>
<h1>You said Rust?</h1>
<p>Yeah, <a href="https://www.rust-lang.org">Rust</a>. It’s a strong and static language aiming at system
programming. Although it’s an <em>imperative</em> language, it has interesting <em>functional</em> conventions
that caught my attention. Because I’m a <em>haskeller</em> and because <em>Rust</em> takes <strong>a lot</strong> from
<em>Haskell</em>, learning it was a piece of cake, even though there are a few concepts I needed a few days
to wrap my mind around. Having a strong C++11/14 experience, it wasn’t that hard though.</p>
<h2>How does it compare to Haskell?</h2>
<p>The first thing that amazed me is the fact that it’s actually not that different from <em>Haskell</em>!
<em>Rust</em> has a powerful type system – not as good as <em>Haskell</em>’s but still – and uses immutability as
a default semantic for bindings, which is great. For instance, the following is forbidden in <em>Rust</em>
and would make <code>rustc</code> – the <em>Rust</em> compiler – freak out:</p>
<pre><code class="language-rust">let a = &quot;foo&quot;;
a = &quot;bar&quot;; // wrong; forbidden
</code></pre>
<p><em>Haskell</em> works like that as well. However, you can introduce mutation with the <code>mut</code> keyword:</p>
<pre><code class="language-rust">let mut a = &quot;foo&quot;;
a = &quot;bar&quot;; // ok
</code></pre>
<p>Mutation should be used only when needed. In <em>Haskell</em>, we have the <code>ST</code> monad, used to introduce
local mutation, or more drastically the <code>IO</code> monad. Under the wood, those two monads are actually
almost the same type – with different warranties though.</p>
<p><em>Rust</em> is strict by default while <em>Haskell</em> is lazy. That means that <em>Rust</em> doesn’t know the concept
of <em>memory suspensions</em>, or <em>thunks</em> – even though you can create them by hand if you want to. Thus,
some algorithms are easier to implement in <em>Haskell</em> thanks to laziness, but some others will
destroy your memory if you’re not careful enough – that’s a very common problem in <em>Haskell</em> due to
thunks piling up in your stack / heap / whatever as you do extensive lazy computations. While it’s
possible to remove those thunks by optimizing a <em>Haskell</em> program – profiling, strictness, etc.,
<em>Rust</em> doesn’t have that problem because it gives you full access to the memory. And that’s a good
thing <strong>if you need it</strong>. <em>Rust</em> exposes <strong>a lot</strong> of primitives to work with memory. In contrast
with <em>Haskell</em>, it doesn’t have a <em>garbage collector</em>, so you have to handle memory on your own. Well,
not really. <em>Rust</em> has several very interesting concepts to handle memory in a very nice way. For
instance, objects’ memory is held by <em>scopes</em> – which have <em>lifetimes</em>.
<a href="https://en.wikipedia.org/wiki/Resource_Acquisition_Is_Initialization">RAII</a> is a very well known
use of that concept and is important in <em>Rust</em>. You can glue code to your type that will be ran when
an instance of that type dies, so that you can clean up memory and scarce resources.</p>
<p><em>Rust</em> has the concept of <em>lifetimes</em>, used to give names to scopes and specify how long an object
reference should live. This is very powerful yet a bit complex to understand in the first place.</p>
<p>I won’t go into comparing the two languages because it would require several articles and a lot of
spare time I don’t really have. I’ll stick to what I’d like to tell you: the <em>Rust</em> implementation
of <em>luminance</em>.</p>
<h1>Porting luminance from Haskell to Rust</h1>
<p>The first very interesting aspect of that port is the fact that it originated from a realization
while refactoring some of my <em>luminance</em> <em>Haskell</em> code. Although it’s functional, stateless and
type-safe, a typical use of <em>luminance</em> doesn’t <em>really</em> require laziness nor a garbage collector.
And I don’t like using a tool – read language – like a bazooka. <em>Haskell</em> is the most powerful
language ever in terms of abstraction and expressivity over speed ratio, but all of that power comes
with an overhead. Even though you’ll find folks around stating that <em>Haskell</em> is pretty okay to code
a video game, I think it will never compete with languages that are <strong>made</strong> to solve real time
computations or reactive programming. And don’t get me wrong: I’m sure you can write a decent video
game in <em>Haskell</em> – I qualify myself as a <em>Haskeller</em> and I’ve not been writing <em>luminance</em> just for
the joy of writing it. However, the way I use <em>Haskell</em> with <em>luminance</em> shouldn’t require all the
overhead – and profiling got me right, almost no GC was involved.</p>
<p>So… I looked into <em>Rust</em> and discovered and learned the language in only three days. I think it’s due
to the fact that <em>Rust</em>, which is simpler than <em>Haskell</em> in terms of type system features and has
almost everything taken from <em>Haskell</em>, is, to me, an <strong>imperative Haskell</strong>. It’s like having a
<em>Haskell</em> minus a few abstraction tools – HKT (but they’ll come soon), GADTs, fundeps, kinds,
constraints, etc. – plus a total control of what’s happening. And I like that. A lot. A <em>fucking</em>
lot.</p>
<p>Porting <em>luminance</em> to <em>Rust</em> wasn’t hard as a <em>Haskell</em> codebase might map almost directly to
<em>Rust</em>. I had to change a few things – for instance, <em>Rust</em> doesn’t have the concept of existential
quantification as-is, which is used intensively in the <em>Haskell</em> version of <em>luminance</em>. But most
<em>Haskell</em> modules map directly to their respective <em>Rust</em> modules. I changed the architecture of
the files to have something clearer. I was working on <em>loose coupling</em> in <em>Haskell</em> for <em>luminance</em>.
So I decided to directly introduce loose coupling into the <em>Rust</em> version. And it works like a
charm.</p>
<p>So there are, currently, two packages available: <code>luminance</code>, which is the core API, exporting the
whole general interface, and <code>luminance-gl</code>, an <strong>OpenGL 3.3</strong> backend – though it will contain more
backends as the development goes on. The idea is that you need both the dependencies to have access
to <em>luminance</em>’s features.</p>
<p>I won’t say much today because I’m working on a <a href="https://en.wikipedia.org/wiki/Demoscene">demoscene</a>
production using <em>luminance</em>. I want it to be a proof that the framework is usable, works and acts
as a first true example. Of course, the code will be open-source.</p>
<p>The documentation is not complete yet but I put some effort documenting almost everything. You’ll
find both the packages here:</p>
<p><a href="https://crates.io/crates/luminance">luminance-0.1.0</a></p>
<p><a href="https://crates.io/crates/luminance-gl">luminance-gl-0.1.0</a></p>
<p>I’ll write another article on how to use <em>luminance</em> as soon as possible!</p>
<p>Keep the vibe!</p>


guid: https://strongly-typed-thoughts.net/blog/luminance-0.7Fri, 13 Nov 2015 00:00:00 GMT
title: OpenGL 3.2 support for luminance!
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance-0.7
feedurl: https://phaazon.net/blog/feed
content:
<h1>luminance-0.7</h1>
<p>You can’t even imagine how hard it was to release <a href="http://hackage.haskell.org/package/luminance-0.7">luminance-0.7</a>.
I came accross several difficulties I had to spend a lot of time on but finally, here it is. I made
a lot of changes for that very special release, and I have a lot to say about it!</p>
<h2>Overview</h2>
<p>As for all my projects, I always provide people with a <em>changelog</em>. The 0.7 release is a major 
release (read as: it was a major increment). I think it’s good to tell people what’s new, but it
should be <strong>mandatory</strong> to warn them about what has changed so that they can directly jump to their
code and spot the uses of the deprecated / changed interface.</p>
<p>Anyway, you’ll find patch, minor and major changes in luminance-0.7. I’ll describe them in order.</p>
<h2>Patch changes</h2>
<h3>Internal architecture and debugging</h3>
<p>A lot of code was reviewed internally. You don’t have to worry about that. However, there’s a new
cool thing that was added internally. It could have been marked as a minor change but it’s not
<em>supposed</em> to be used by common people – you can use it via a flag if you use <code>cabal</code> or <code>stack</code>
though. It’s about debugging the <em>OpenGL</em> part used in luminance. You shouldn’t have to use it but
it could be interesting if you spot a bug someday. Anyway, you can enable it with the flag
<code>debug-gl</code>.</p>
<h3>Uniform Block / Uniform Buffer Objects</h3>
<p>The <a href="https://www.opengl.org/wiki/Uniform_Buffer_Object">UBO</a> system was buggy and was fixed. You
might experience issue with them though. I spotted a bug and reported it – you can find the bug
report <a href="https://bugs.freedesktop.org/show_bug.cgi?id=92909">here</a>. That bug is not Haskell related
and is related to the i915 Intel driver.</p>
<h2>Minor changes</h2>
<p>The minor changes were the most important part of luminance-0.7. luminance now officially supports
<em>OpenGL 3.2</em>! When installing luminance, you default to the <code>gl32</code> backend. You can select the
backend you want with flags – <code>gl45</code> and <code>gl45-bindless-textures</code> – but keep in mind you need the
appropriate hardware to be able to use it. Because you need to use flags, you won’t be able to
switch to the backend you want at runtime – that’s not the purpose of such a change though.</p>
<p>The performance gap should be minor between <code>gl32</code> and <code>gl45</code> but still. Basically, OpenGL 4.5 adds
the support for <a href="https://www.opengl.org/wiki/Direct_State_Access">DSA</a>, which is very handy and less
ill-designed that previous iterations of OpenGL. So a lot of code had to be rewritten to implement
luminance’s stateless interface without breaking performance nor bring them down.</p>
<p>I <em>might</em> add support for other backends later on – like an <em>OpenGL ES</em> backend and <em>WebGL</em> one –
but that won’t ship that soon though because I have a ton of work to do, and yet need to provide you
with a concrete, beautiful, fast, appealing and eye-blowing demo with luminance! ;)</p>
<p>Feel free to test the <code>gl32</code> backend and give me back feedback!</p>
<p>However, if I spent so much time on that 0.7 version, it’s because I had issue whilst writing the
<code>gl32</code> backend. Indeed, I spotted several bugs on my Intel HD card. This is my OpenGL version string
for my Intel IGP card:</p>
<blockquote>
<p>OpenGL core profile version string: 3.3 (Core Profile) Mesa 11.0.4</p>
</blockquote>
<p>The architecture is Haswell. And on such a card (<code>i915</code> linux driver) I’ve found two bugs while
trying the <code>gl32</code> backend with <a href="https://hackage.haskell.org/package/luminance-samples-0.7">luminance-samples-0.7</a>.</p>
<h3><code>usampler2D</code></h3>
<p>For unknown reason, the <code>Texture</code> sample failed on my Intel IGP but ran smoothly and flawlessly on
my nVidia GPU. I spent a lot of time trying to figure out what I was missing, but eventually changed
the sampler type – it’s now a <code>sampler2D</code> – and… it worked. I reported the issue to the intel dev
team. So if you hit that error too, please leave a message here so that I can have more hindsight
about that error and see what I can do.</p>
<h3>Uniform block and <code>vec3</code></h3>
<p>This is a very nasty issue that kept me awoken for days trying to fix <strong>my</strong> code while it was a
driver bug. It’s a big technical, so I’ll just leave a <a href="https://bugs.freedesktop.org/show_bug.cgi?id=92909">link to the bug tracker</a>
so that you can read it if you want to.</p>
<h2>Breaking changes</h2>
<p>Ok, let’s talk.</p>
<p>When creating a new shader stage, you now have to use the function <code>createStage</code> – instead of
several functions like <code>createVertexShader</code>. That change is important so that I can add new shader
types without changing the interface, and because some shader can fail to be created. For instance,
on the <code>gl32</code> backend, trying to build a tessellation shader will raise an error.</p>
<p>When a shader stage creation fails, the <code>UnsupportedStage</code> error is raised and holds the type of the
stage that failed.</p>
<p>Finally, the interface for the cubemaps changed a bit – you don’t have access to <em>width</em> and
<em>height</em> anymore, that was error-prone and useless; you’re stick to a <em>size</em> parameter.</p>
<p>I’d like to thank all people supporting me and luminance. I’ll be watching reactions to that major
and important release as it will cover more people with cheaper and well-spread GPUs.</p>
<p>Happy hacking! :)</p>


guid: https://strongly-typed-thoughts.net/blog/luminance_0.1Tue, 22 Sep 2015 00:00:00 GMT
title: luminance 0.1 released!
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance_0.1
feedurl: https://phaazon.net/blog/feed
content:
<h1>Here we are</h1>
<p><a href="http://hackage.haskell.org/package/luminance-0.1">luminance-0.1</a> was released yesterday night,
along with <a href="http://hackage.haskell.org/package/luminance-samples-0.1">luminance-samples-0.1</a>! I’ll
need to enhance the documentation and add directions so that people don’t feel too overwhelmed.</p>
<p>I’m also going to write a wiki to help people get their mind wrapped around <strong>luminance</strong>.</p>
<p>If you think something is missing; if you think something could be enhanced; or if you’ve found a
bug, please, feel free to fill in an issue on the
<a href="https://github.com/phaazon/luminance/issues">issues tracker</a>.</p>
<h1>Next big steps</h1>
<p>I need to test the framework. I need a <em>lot</em> of tests. I’ll write a demoscene production with it so
that I can give a good feedback to the community and prove that <strong>luminance</strong> can be used and works.</p>
<p>In the waiting, keep the vibe!</p>


guid: https://strongly-typed-thoughts.net/blog/smoothieThu, 26 Feb 2015 00:00:00 GMT
title: Smoothie, a Haskell library for creating smooth curves
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/smoothie
feedurl: https://phaazon.net/blog/feed
content:
<h1>The smoother the better!</h1>
<p>It’s been a while I haven’t written anything on my blog. A bit of refreshment
doesn’t hurt much, what do you think?</p>
<p>As a <a href="http://en.wikipedia.org/wiki/Key_frame">demoscener</a>, I attend
demoparties, and there will be a very important and fun one in about a month.
I’m rushing on my 3D application so that I can finish something to show up, but
I’m not sure I’ll have enough spare time. That being said, I need to be able to
represent smooth moves and transitions without any tearing. I had a look into a
few <strong>Haskell</strong> <em>spline</em> libraries, but I haven’t found anything
interesting – or not discontinued.</p>
<p>Because I do need splines, I decided to write my very own package. Meet
<a href="https://hackage.haskell.org/package/smoothie">smoothie</a>, my <strong>BSD3</strong> Haskell
spline library.</p>
<h2>Why splines?</h2>
<p>A <em>spline</em> is a curve defined by several polynomials. It has several uses, like
vectorial graphics, signal interpolation, animation tweening or simply plotting
a spline to see how neat and smooth it looks!</p>
<p>Splines are defined using polynomials. Each polynomials is part of the curve and
connected one-by-one. Depending on which polynomial(s) you chose, you end up
with a different shape.</p>
<p>For instance, 1-degree polynomials are used to implement straight lines.</p>
<p><img src="http://phaazon.net/pub/linear_spline.png" alt="" /></p>
<p>As you can see, we can define a few points, and interpolate in between. This is
great, because we can turn a <em>discrete</em> set of points into lines.</p>
<p>Even better, we could use 3-degree polynomials or cosine functions to make each
part of the spline smoother:</p>
<p><img src="http://phaazon.net/pub/spline.png" alt="" /></p>
<p>We still have discrete points, but in the end, we end up with a smooth set of
points. Typically, imagine sampling from the spline with time for a camera
movement. It helps us to build smooth moves. This is pretty important when doing
animation. If you’re curious about that, I highly recommend having a look into
<a href="http://en.wikipedia.org/wiki/Key_frame">key frames</a>.</p>
<h2>Using splines in Haskell</h2>
<p>So I’ve been around implementing splines in Haskell the most general way as
possible. However, I don’t cover – yet? – all kinds of splines. In order to
explain my design choices, I need to explain a few very simple concepts first.</p>
<h3>Sampling</h3>
<p>A spline is often defined by a set of points and polynomials. The first point
has the starting <em>sampling</em> value. For our purpose, we’ll set that to <em>0</em>:</p>
<pre><code class="language-haskell">let startSampler = 0
</code></pre>
<p>The sampling value is often <code>Float</code>, but it depends on your spline and the use
you want to make of it. It could be <code>Int</code>. The general rule is that it <strong>should
be orderable</strong>. If we take two sampling values <code>s</code> and <code>t</code>, we should be able
to compare <code>s</code> and <code>t</code> (that’s done through the typeclass constraint <code>Ord</code> in
Haskell).</p>
<p>So, if you have a spline and a sampling value, the idea is that sampling the
spline with <code>startSampler</code> gives you the first point, and sampling with
<code>t</code> with <code>t &gt; startSampler</code> gives you another point, interpolated using points
of the spline. It could use two points, three, four or even more. It actually
depends on the polynomials you use, and the interpolating method.</p>
<p>In <a href="https://hackage.haskell.org/package/smoothie">smoothie</a>, sampling values
have types designed by <code>s</code>.</p>
<h3>Control points</h3>
<p>A spline is made of points. Those points are called <strong>control points</strong> and
<a href="https://hackage.haskell.org/package/smoothie">smoothie</a> uses <code>CP s a</code> to refer
to them, where <code>s</code> is the sampling type and <code>a</code> the carried value.</p>
<p>Although they’re often used to express the fact that the curve should pass
through them, they don’t have to lie on the curve itself. A very common and
ultra useful kind of spline is the B-spline.</p>
<p><img src="http://phaazon.net/pub/b_spline.png" alt="" /></p>
<p>With that kind of spline, the property that the curve passes through the control
points doesn’t hold. It passes through the first and last ones, but the ones
in between are used to <em>shape</em> it, a bit like <strong>magnets</strong> attract things around
them.</p>
<p>Keep in mind that <strong>control points</strong> are very important and used to define the
main aspect of the curve.</p>
<h3>Polynomials</h3>
<p>Polynomials are keys to spline interpolation. They’re used to <em>deduce</em> sampled
points. Interpolation is a very general term and used in plenty of domains. If
you’re not used to that, you should inquiry about
<a href="http://en.wikipedia.org/wiki/Linear_interpolation">linear interpolation</a> and
<a href="http://en.wikipedia.org/wiki/Cubic_Hermite_spline">cubic interpolation</a>, which
are a very good start.</p>
<p>Polynomials are denoted by <code>Polynomial s a</code> in
<a href="https://hackage.haskell.org/package/smoothie">smoothie</a>, where <code>s</code> and <code>a</code> have
the same meaning than in <code>CP s a</code>.</p>
<h2>Getting started with smoothie</h2>
<h3>Types and constraints</h3>
<p><a href="https://hackage.haskell.org/package/smoothie">smoothie</a> has then three important
types:</p>
<ul>
<li><code>CP s a</code>, the control points</li>
<li><code>Polynomial</code>, the polynomials used to interpolate between control points</li>
<li><code>Spline s a</code>, of course</li>
</ul>
<p>The whole package is parameterized by <code>s</code> and <code>a</code>. As said earlier, <code>s</code> is very
likely to require an <code>Ord</code> constraint. And <code>a</code>… Well, since we want to represent
points, let’s wonder: which points? What kind of points? Why even <em>“points”</em>?
That’s a good question. And this is why you may find
<a href="https://hackage.haskell.org/package/smoothie">smoothie</a> great: it doesn’t
actually know anything about points. It accepts <strong>any kind of values</strong>. Any?
Almost. Any values that are in an <strong>additive group</strong>.</p>
<blockquote>
<p><em>“What the…”</em></p>
</blockquote>
<p>I won’t go into details, I’ll just vulgarize them so that you get quickly your
feet wet. That constraint, when applied to <strong>Haskell</strong>, makes <code>a</code> to be
an endofunctor – i.e. <code>Functor</code> – and additive – i.e. <code>Additive</code>. It also
requires it to be a first-class value – i.e. its kind should be <code>* -&gt; *</code>.</p>
<p>With <code>Functor</code> and <code>Additive</code>, we can do two important things:</p>
<ul>
<li>First, with <code>Functor</code>. It enables us to lift computation on the inner type.
We can for instance apply a single function inside <code>a</code>, like <code>*k</code> or <code>/10</code>.</li>
<li>Then, with <code>Additive</code>. It enables us to add our types, like <code>a + b</code>.</li>
</ul>
<p>We can then make <strong>linear combinations</strong>, like a<em>k + b</em>q. This property is well
known for <a href="http://en.wikipedia.org/wiki/Vector_space">vector spaces</a>.</p>
<p>The fun consequence is that providing correct instances to <code>Functor</code> and
<code>Additive</code> will make your type useable with
<a href="https://hackage.haskell.org/package/smoothie">smoothie</a> as carried value in the
spline! You might also have to implement <code>Num</code> and <code>Ord</code> as well, though.</p>
<h3>Creating a spline</h3>
<p>Creating a spline is done with the <code>spline</code> function, which signature is:</p>
<pre><code class="language-haskell">spline :: (Ord a, Ord s) =&gt; [(CP s a, Polynomial s a)] -&gt; Spline s a
</code></pre>
<p>It takes a list of <strong>control points</strong> associated with <strong>polynomials</strong> and
outputs a <strong>spline</strong>. That requires some explainations… When you’ll be sampling
the spline,
<a href="https://hackage.haskell.org/package/smoothie">smoothie</a> will look for which
kind of interpolation method it has to use. This is done by the lower nearest
control point to the sampled value. Basically, a pair <code>(cp,polynomial)</code> defines
a new point and the interpolation method to use for the curve ahead of the
point.</p>
<p>Of course, the latest point’s polynomial won’t be used. You can set whatever you
want then – protip: you can even set <code>undefined</code> because of laziness.</p>
<p>Although the list will be sorted by <code>spline</code>, I highly recommend to pass a
sorted list, because dealing with unordered points might have no sense.</p>
<p>A control point is created by providing a sample value and the carried value.
For instance, using <a href="https://hackage.haskell.org/package/linear">linear</a>’s <code>V2</code>
type:</p>
<pre><code class="language-haskell">let cp0 = CP 0 $ V2 1 pi
</code></pre>
<p>That’s a control point that represents <code>V2 1 pi</code> when sampling is at <code>0</code>. Let’s
create another:</p>
<pre><code class="language-haskell">let cp1 = CP 3.341 $ V2 0.8 10.5
</code></pre>
<p>Now, let’t attach a polynomial to them!</p>
<h4>Hold that for me please</h4>
<p>The simplest polynomial – wich is actually not a polynomial, but heh, don’t look
at me that way – is the 0-degree polynomial. Yeah, a constant function. It takes
the lower control point, and <em>holds</em> it everwhere on the curve. You could
picture that as a staircase function:</p>
<p><img src="http://phaazon.net/pub/hold_spline.gif" alt="" /></p>
<p>You might say that’s useless; it’s actually not; it’s even pretty nice. Imagine
you want to attach your camera position onto such a curve. It will make the
camera <em>jump</em> in space, which could be desirable for flash looks!</p>
<p>Use the <code>hold</code> <code>Polynomial</code> to use such a behavior.</p>
<h4>Oh yeah, it’s straightforward!</h4>
<p>1-degree functions often describe <em>lines</em>. That is, <code>linear</code> is the <code>Polynomial</code>
to use to connect control points with… straight lines.</p>
<h4>Have fun on the unit circle!</h4>
<p>One very interesting <code>Polynomial</code> is <code>cosine</code>, that defines a cosine
interpolation, used to smooth the spline and make it nicer for moves and
transitions.</p>
<h4>Highway to the danger zone</h4>
<p>If you’re crazy, you can experiment around with <code>linearBy</code>, which, basically, is
a 1-degree polynomial if you pass <code>id</code>, but will end up in most complex shapes
if you pass another function – <code>(s -&gt; s)</code>. Dig in documentation on <em>hackage</em> for
further information.</p>
<h4>Sampling our spline</h4>
<p>Ok, let’s use a linear interpolation to sample our spline:</p>
<pre><code class="language-haskell">let spl = spline [(cp0,linear),(cp1,hold)]
</code></pre>
<blockquote>
<p><em>Note: I used <code>hold</code> as a final polynomial because I don’t like using
<code>undefined</code>.</em></p>
</blockquote>
<p>Ok, let’s see how to sample that.
<a href="https://hackage.haskell.org/package/smoothie">smoothie</a> exports a convenient
function for sampling:</p>
<pre><code class="language-haskell">smooth :: Ord s =&gt; Spline s a -&gt; s -&gt; Maybe a
</code></pre>
<p><code>smooth spl s</code> takes the sampling value <code>s</code> and maybe interpolate it in the
<code>spl</code> spline.</p>
<blockquote>
<p><em>“Maybe? Why aren’t you sure?”</em></p>
</blockquote>
<p>Well, that’s pretty simple. In some cases, the curve is not defined at the
sampling value you pass. Before the first point and after, basically. In those
cases, you get <code>Nothing</code>.</p>
<h2>That’s not the end</h2>
<p>I wrote <a href="https://github.com/phaazon/smoothie">smoothie</a> in a few hours, in a
single day. You might have ideas. I want it to be spread and widely used by
awesome people. Should you do graphics programming, sound programming, animation
or whatever implying splines or smoothness, please provide feedback!</p>
<p>For people that would like to get contributing, here’s the
<a href="https://github.com/phaazon/smoothie">github page</a> and the
<a href="https://github.com/phaazon/smoothie/issues">issue tracker</a>.</p>
<p>If no one comes up with, I’ll try to add some cubic interpolation methods, like
<em>hermitian splines</em>, and one of my favorite, the famous <strong>Catmull Rom</strong> spline
interpolation method.</p>
<p>As always, have fun hacking around, and keep doing cool stuff and sharing it!</p>


guid: https://strongly-typed-thoughts.net/blog/type_aliasingWed, 07 Jan 2015 00:00:00 GMT
title: Why has type aliasing almost killed me
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/type_aliasing
feedurl: https://phaazon.net/blog/feed
content:
<h1>Type aliasing</h1>
<p>Type aliasing is something that’s been included in most languages for a while
now. Basically, if you have a type <code>Foo</code>, aliasing <code>Bar</code> to <code>Foo</code> means that
you introduce a new <em>symbol</em> <code>Bar</code> that refers to <code>Foo</code>. It’s not a new type,
it’s just an alias, a <em>binding</em>.</p>
<p>In <strong>Haskell</strong>, we do aliasing with the keyword <code>type</code>, which is a bit stupid
since we won’t necessarily alias a type (we could alias a function, a typeclass
constraint, and so on and so forth).</p>
<p>Aliasing is great and useful in many ways. For instance, there’s a Haskell
package I love called <a href="https://hackage.haskell.org/package/linear">linear</a>. That
package is used to deal with linear algebra, and you can find common math types
in there among vectors (<code>Vec</code>) and low-algebra vectors (<code>V2</code>, <code>V3</code>, <code>V4</code>). There
are the same things for matrices (<code>M22</code> for <em>2x2</em> matrices, <code>M33</code>, <code>M44</code>).</p>
<p>If you come up with using <em>4x4</em> floating matrices a lot, you might end up with a
lot of <code>M44 Float</code> everywhere in your code. So that would be sane aliasing it.
Like so:</p>
<pre><code class="language-haskell">type V4F = V4 Float
</code></pre>
<p>This is a perfect aliasing use to me as <code>V4F</code> is straight-forward for
<em>graphics</em> people. Furthermore, <code>linear</code> is pretty a standard now in Haskell
linear algebra application, so <code>V4</code> should be familiar to you.</p>
<p>However, there is a drawback with type aliasing.</p>
<h1>Why has type aliasing almost killed me</h1>
<p>I spent more than a day crawling for a bug. I was refactoring code, a lot of
code. Nothing fancy though, just some code cleaning. Nevertheless, I created a
brand new git branch to make my cleaning. On the initial branch, the application
behavior was normal. After having cleaned the code, I got a black screen.</p>
<p>After a long investigation, I found out that the bug was due to a mismatch
between two matrices – i.e. <code>M44 Float</code>. One matrix is used as a camera
projection and the other one is used as a light projection. In term of code,
both matrices are placed in a type. I simplified (a lot), but this is the idea:</p>
<pre><code class="language-haskell">data Foo = Foo {
    -- […]
  , cameraProjection :: M44 Float
  , lightProjection :: M44 Float
  }
</code></pre>
<p>What makes those matrices different in term of compilation? Nothing.
<code>M44 Float</code> and <code>M44 Float</code> are the same type. Swapping matrices is then
silently compiled, since it’s not an error to the compiler. And such a swapping
result in a terrible and hard to find bug.</p>
<blockquote>
<p><em>Yeah, so we could have simply aliased those matrices to two different names!</em></p>
</blockquote>
<p>Remember what I said earlier. Aliased types are not new types. They’re bindings
in the type system:</p>
<pre><code class="language-haskell">type CameraProjection = M44 Float
type LightProjection = M44 Float
</code></pre>
<p>Here, both <code>CameraProjection</code> and <code>LightProjection</code> are the same type. If a
function expects a <code>CameraProjection</code>, it actually expects a <code>M44 Float</code>. You
could then pass a value of type <code>M44 Float</code> or even <code>LightProjection</code>, that
would be legit.</p>
<blockquote>
<p><em>Is there a solution yet?</em></p>
</blockquote>
<p>Yes, there is. <strong>Haskell</strong> has several keywords to create types:</p>
<ul>
<li><code>type</code>, which is, as we’ve already seen, a way to alias types;</li>
<li><code>data</code>, which is used to create data types;</li>
<li><code>newtype</code>, which is used as a type wrapper to introduce a data type.</li>
</ul>
<p><code>newtype</code> is the most important keyword for us here. Basically, it has the same
semantic as <code>data</code> except that <code>newtype</code> introduces a type with a single
<em>field</em>; it can’t have none nor several. <code>newtype</code> is often used to express the
<em>same thing</em> that the inner type, but adding / removing some properties.</p>
<p>For instance, <code>Bool</code> can’t be an instance of <code>Monoid</code> since we could define a
lot of instances. <code>mempty</code> could be <code>True</code> if we consider the monoid of
booleans with <strong>AND</strong> but it would be <code>False</code> if we consider the monoid of
booleans with <strong>OR</strong>. Well, we could write those, actually:</p>
<pre><code class="language-haskell">newtype And = And { getAnd :: Bool } deriving WhateverYouWant
newtype Or = Or { getOr :: Bool } deriving WhateverYouWant

instance Monoid And where
  mempty = True
  And a `mappend` And b = And (a &amp;&amp; b)

instance Monoid Or where
  mempty = False
  Or a `mappend` Or b = Or (a || b)
</code></pre>
<p>Those two types are defined in <code>Data.Monoid</code> as <code>All</code> and <code>Any</code>.</p>
<p>Back to our problem. We could use <code>newtype</code> to add <code>M44 Float</code> the correct
semantic, and disable us to mix them:</p>
<pre><code class="language-haskell">newtype CameraProjection = CameraProjection { getCameraProjection :: M44 Float }
newtype LightProjection = LightProjection { getLightProjection :: M44 Float }
</code></pre>
<p>And here, <code>CameraProjection</code> ≠ <code>LightProjection</code>. If a function expects a
<code>CameraProjection</code>, you can’t pass anything else than a <code>CameraProjection</code>.</p>
<p>However, be careful. You’ll have to wrap a <code>M44 Float</code> into your
<code>CameraProjection</code>. Such a function, like the <code>CameraProjection</code> constructor, is
not semantic safe since you could still pass a light representation.
Nevertheless, it’s better than the initial design since once wrapped, you can’t
make confusions anymore.</p>
<h1>Final words</h1>
<p>To sum up, use aliasing when it’s handy but don’t use it if it could break
semantics or add confusion. If you have several types that use the same
type (e.g. colors and positions could use the same <code>V3 Float</code> type), <strong>don’t
use aliases</strong>! Create proper types to prevent confusion later on. Your compiler
will reward you, then you don’t get nasty runtime bugs ;).</p>


guid: https://strongly-typed-thoughts.net/blog/luminance_32bitTue, 11 Aug 2015 00:00:00 GMT
title: Luminance – what was that alignment stuff already?
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance_32bit
feedurl: https://phaazon.net/blog/feed
content:
<p>Yesterday, I released a new article about how I implement vertex arrays in
luminance. In that article, I told you that the memory was packed with
alignment set to <strong>1</strong>.</p>
<p>Well, I’ve changed my mind. Some people pointed out that the good thing to do
for most GPU is to align on 32-bit. That is, <strong>4</strong> bytes. The alignment should
be <strong>4</strong> bytes, then, not <strong>1</strong>.</p>
<p>There might be an issue with that. If you store a structure with attributes
which sizes are not a multiple of <strong>4</strong> bytes, it’s likely you need to add
padding.</p>
<p>However, I just reviewed my code, and found this:</p>
<pre><code class="language-haskell">instance (GPU a,KnownNat n,Storable a) =&gt; Vertex (V n a) where
instance (Vertex a,Vertex b) =&gt; Vertex (a :. b) where
</code></pre>
<p>Those are the single instances for <code>Vertex</code>. That means you can only use <code>V</code>
and <code>(:.)</code> to build up vertices. Look at the <code>V</code> instance. You’ll find a <code>GPU</code>
typeclass constraint. Let’s look at its definition and instances:</p>
<pre><code class="language-haskell">class GPU a where
  glType :: Proxy a -&gt; GLenum

instance GPU Float where
  glType _ = GL_FLOAT

instance GPU Int32 where
  glType _ = GL_INT

instance GPU Word32 where
  glType _ = GL_UNSIGNED_INT

</code></pre>
<p>Woah. How did I forget that?! Let me translate those information to you. That
means we can only have 32-bit vertex component! So the memory inside vertex
buffers will always be aligned on <strong>4</strong> bytes! No need to worry about padding
then!</p>
<p>The first implication is the fact you won’t be able to use <code>Word16</code>, for
instance. You’ll need to stick to the three types that have a <code>GPU</code> instance.</p>
<p><strong>Note</strong>: that doesn’t prevent us from adding <code>Double</code> later on, because a
<code>Double</code> is a 64-bit type, which is a multiple of <strong>4</strong> bytes!</p>
<p>That’s all I have for today. I’m working on something very exciting linked to
render batching. I’ll talk about that when it’s cooked. ;)</p>
<p>Keep the vibe; keep building awesome things, and as always, thank you for
reading me!</p>


guid: https://strongly-typed-thoughts.net/blog/al_0.1.0.2Sun, 01 Mar 2015 00:00:00 GMT
title: al 0.1.0.2 – documentation and default paths
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/al_0.1.0.2
feedurl: https://phaazon.net/blog/feed
content:
<h1>al patch</h1>
<p>This is a very short article to make you notice that
<a href="https://hackage.haskell.org/package/al">al</a> received two important changes:</p>
<ul>
<li>I uploaded documentation (hourra!) ;</li>
<li>OpenAL paths will default to default installation on <strong>Windows systems</strong>.</li>
</ul>
<p>I tested the latter with a Windows 64b:</p>
<pre><code>cabal update
cabal install al
</code></pre>
<p>That’s all. You should try it out as well. If you have errors about OpenAL
libraries and/or header files, check whether OpenAL is correctly installed.
If the error comes up again, proceed as I said
<a href="http://phaazon.blogspot.fr/2015/02/al-01-released.html#installing-al-1">here</a>.</p>
<p>Also, I take people on linux feedback about installs ;).</p>


