# rss_item

guid: https://strongly-typed-thoughts.net/blog/c++-constructorsMon, 24 Feb 2020 00:35:00 GMT
title: Let’s talk about C++ constructors
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/c++-constructors
feedurl: https://phaazon.net/blog/feed
content:
<p>Lately, I had an interesting talk with colleagues of mine about C++. I told them I needed a safe
<code>std::unique_ptr</code> abstraction. I needed such a type because I was handling scarce resources that
needed to be uniquely owned in our code base, but at the same time, checking every time whether
a value is present (<code>nullptr</code> being the problem here) yields large numbers of bugs in production.</p>
<p>They asked <em>“What’s a safest <code>std::unique_ptr</code>?”</em> My main problem was (still is) that
<code>std::unique_ptr</code> can be initialized with <code>nullptr</code>. The problem with this is that you cannot
assume that you <em>always</em> own a <code>T</code> if you are given a <code>std::unique_ptr&lt;T&gt;</code>. References and wrapped
references cannot be used either because we really want to own the data here.</p>
<p>The rest of the discussion is driven by that initial motivation.</p>
<p>As a (mainly) Haskell and Rust developer, I’m also pretty good at writing C and C++ — after all,
I’ve been doing C and C++ since I’m 11 and today, I’m 28; I’ll let you do the math. I especially
developed 3D stuff, ranging from toy experiments to demoscene productions, and lots of other
low-levels things I won’t detail here.</p>
<p>See, I’ve failed a lot while trying things when I was younger — especially around the age of
15 / 16. That’s basically the age where I started to be <em>very</em> frustrated at C++ and, without even
knowing it, was asking to find a <em>better way to design code</em>. Yes, I failed a lot, I have no shame
stating it. But to me, failing has been a building block to my experience. The more I fail, the less
likely I’ll fail again the same way. By definition, if you fail a lot on different things… you now
have very good hints and ideas about how <em>not to fail again</em>. This is like <em>learn the hard way</em>, but
I truly think it’s important, especially to get <em>why</em> something doesn’t work. Reading books and
being told is important, but failing is even more.</p>
<p>Yes, today, with all the hindsight I’ve been through, all the languages I’ve been using – ranging
from C/C++, D, Python, Haskell, Idris, OCaml, Rust, Java, JavaScript, Perl, C#, GLSL and several
others — I think I have a pretty good idea about what’s out there in terms of language ideas.</p>
<p>The goal of this article is to explain <em>exactly why</em> I don’t like that much C++, <em>why</em> I think it
has a lot of bad ideas, and to provide people with material to think and meditate. I truly think
that a good developer needs to call themselves into question regularly and question their process
and established knowledge. I think that, decades ago, C and C++ were the right way to do things.
Today, I think those languages and especially <em>what has been accepted as ground and established
truth</em> about how to do things has become obsolete.</p>
<p>I’m going to explain why in a series of blog articles, starting with this one. Let’s go.</p>
<!-- vim-markdown-toc GFM -->
<ul>
<li><a href="#c-and-the-wrong-idea-of-constructors">C++ and the wrong idea of constructors</a>
<ul>
<li><a href="#hidden-costs">Hidden costs</a></li>
<li><a href="#the-c-broken-initialization-design">The C++ broken initialization design</a></li>
<li><a href="#construction--constructors">Construction ≠ constructors</a></li>
</ul>
</li>
<li><a href="#c-and-its-bad-standard-library">C++ and its bad standard library</a></li>
<li><a href="#how-would-you-do-that-in-rust">How would you do that in Rust?</a></li>
</ul>
<!-- vim-markdown-toc -->
<blockquote>
<p><a href="https://phaazon.net/blog/c++-exceptions">Next article in the series</a>.</p>
</blockquote>
<h1>C++ and the wrong idea of constructors</h1>
<p>I want to start with a very good example of something that has been accepted as pretty nice when
it was introduced and that is <em>completely</em> obsolete to me. Let’s talk about C++’s <em>constructors</em>.
But not <em>constructors</em> by themselves. I want to focus on <em>how most people</em> use them, especially
around the concept of <em>fallible construction</em> — we’re going to talk about exceptions, too!</p>
<p>For those who are not familiar with constructors in C++, let me explain a bit. In C, when you have
a data type (like a <code>struct</code>), you can create an object with such a type by allocating memory for it
(either on the stack or the heap) and then initialize its fields. Since C99, you also have
the <em>designated initializer</em> syntax to initialize a <code>struct</code> directly while you’re declaring it.
Some examples:</p>
<pre><code class="language-c">struct Foo {
  int x;
  char const* y;
};

int main() {
  Foo foo;
  foo.x = 3;
  foo.y = &quot;Hello, world!&quot;;

  // designated initializer syntax
  Foo foo2 = {
    .x = 3,
    .y = &quot;Hello, world!&quot;
  };
}
</code></pre>
<p>Even though that code seems pretty <em>harmless</em>, to me, it’s very easy to misuse C’s construction
syntax. I will not explain exactly why here because we’re interested in C++, but you’ll understand
what’s the problem by reading on. It relates to being possible to leave a state or part of a state
undefined without having your compiler prevent you from doing that.</p>
<h2>Hidden costs</h2>
<p>In C++, you cannot go with any of those way to create a value without a mandatory C++ concept:
<em>constructors</em>. Constructors are C++’s way to initialize objects by using the same kind of syntax.
C++ has several constructors:</p>
<ul>
<li><em>Default constructor</em>. This one is invoked when you want to create an object without constructing
it with any parameter. It’s an important constructor in C++, as several containers and standard
algorithms rely on its presence.</li>
<li><em>Parameterized constructors</em>: you can create as many parameterized constructors as you want and
they will take parameters to initialize your objects based on those parameters.</li>
<li><em>Copy constructors</em>: special constructors called when you want to copy objects. In C++, a
copy constructor is automatically called when you pass an object <code>foo</code> which type is <code>Foo</code> to a
function that expects a <code>Foo</code>, for instance.</li>
<li><em>Move constructors</em>: special constructors called when you <em>move</em> objects around. In C++, a move
constructor is automatically called when you pass a rvalue reference to something (<code>Foo &amp;&amp;</code>).
You can obtain such references by using <code>std::move</code> or <code>std::forward</code>.</li>
</ul>
<blockquote>
<p>There is a slightly exception: <a href="https://en.cppreference.com/w/cpp/language/aggregate_initialization">aggregate initialization</a>, which is a bit similar to the C’s
designated initialization, but requires that you have not declared any constructor – among other
restrictions — and is then always available, either for public or private code. I will then not
talk about it. I also have to admit I never completely recall all the rules about POD designated
initialization.</p>
</blockquote>
<p>Let’s demonstrate some of the constructors from above in the following program:</p>
<pre><code class="language-cpp">#include &lt;iostream&gt;

struct Foo {
  // default constuctor
  Foo() {
    std::cout &lt;&lt; &quot;default ctor called&quot; &lt;&lt; std::endl;
  }

  // copy constructor
  Foo(Foo const &amp;) {
    std::cout &lt;&lt; &quot;copy ctor called&quot; &lt;&lt; std::endl;
  }

  // move constructor
  Foo(Foo &amp;&amp;) {
    std::cout &lt;&lt; &quot;move ctor called&quot; &lt;&lt; std::endl;
  }

  // assignment operator by copy
  Foo &amp; operator=(Foo const &amp;) {
    std::cout &lt;&lt; &quot;operator= called by copy&quot; &lt;&lt; std::endl;
    return *this;
  }

  // assignment operator by move
  Foo &amp; operator=(Foo &amp;&amp;) {
    std::cout &lt;&lt; &quot;operator= called by move&quot; &lt;&lt; std::endl;
    return *this;
  }
};

Foo get_foo() {
  return Foo();
}

int main() {
  Foo foo;
  auto foo2 = foo;
  auto foo3 = get_foo();

  return 0;
}
</code></pre>
<p>In the <code>main</code>, what constructors do you think each line is going to call? :) The answer is… it
depends on your compiler. Yes, you heard me right. With my compiler, this is what I get:</p>
<ul>
<li><code>Foo foo;</code>: default constructor, which should be the case for every compiler here.</li>
<li><code>auto foo2 = foo;</code>: copy constructor.</li>
<li><code>auto foo3 = get_foo();</code>: default constructor. This is due to something called <a href="https://en.cppreference.com/w/cpp/language/copy_elision">(N)RVO</a>.</li>
</ul>
<p>This is already pretty bad, for several reasons:</p>
<ul>
<li>It’s hard to tell when we are going to copy something. Think of a more complex data structure,
such as a <code>std::vector&lt;Something&gt;</code>, which might contain thousands of millions of <code>Something</code>.
Because C++ copies by default with the <em>value semantics</em>, you might end up with several copies
while your code doesn’t explicitly copy anything.</li>
<li>Because copy constructors and move constructors are subject to compiler optimizations, we don’t
really know when the <a href="https://en.cppreference.com/w/cpp/language/copy_elision">(N)RVO</a> optimization is going to kick in — we can have good confidence
but we might be wrong. I know some people never really know what <code>auto foo = something;</code> does —
and I feel them, I often spend some time trying to figure out as well.</li>
</ul>
<p>The existence of copy constructors and the value semantics by default makes C++ hard to reason
about when we are making copies. Other languages, such as Rust, mitigate that risk by using a
move semantics by default. If you want to copy (i.e. clone) something, you have to explicitly ask
for it. For instance, consider:</p>
<pre><code class="language-cpp">// C++
void foo(std::vector&lt;Foo&gt; foos) {
  // …
}

int main() {
  std::vector&lt;Foo&gt; foos;

  // …

  foo(foos);

  return 0;
</code></pre>
<p>vs.</p>
<pre><code class="language-rust">fn foo(foos: Vec&lt;Foo&gt;) {
  // …
}

fn main {
  let foos = Vec::new();

  // …

  foo(foos);
}
</code></pre>
<p>The C++ program will copy the content of <code>foos</code> when calling <code>foo</code>. The copy here is due to the
copy constructor of <code>std::vector</code> being called. The effect of is that the whole content of the
heap-allocated region owns by the vector will be copied. I typically call that a <em>deep copy</em>, or
simply a <em>clone</em>.</p>
<p>The Rust program will not make a copy of <code>foos</code>. Instead, it will move ownership from the caller
to the calle. <code>foos</code> won’t be available anymore in <code>main</code> after the call to <code>foo</code>. So no heap
allocation will occur here. If we wanted the same behavior as the C++ one, we would do this:</p>
<pre><code class="language-rust">  foo(foos.clone());
</code></pre>
<p>That requires that <code>Foo</code> implement the <code>Clone</code> trait, but it’s off topic here. However, there a
few places where Rust can still do <em>hidden</em> copies that can hurt your performance. By default,
everything is moved. However, if your type implements the <code>Copy</code> trait, it means that it can be
safely copied. Such a copy is always performed bit-wise. What’s interesting is that there is, in
theory, nothing different at runtime between <em>copying</em> and <em>moving</em> in Rust. Moving might allow
more optimizations to prevent actually copying data, but imagine you don’t have optimizations.
Both copy and move semantics, in Rust, could be implemented with a <code>memcpy</code>. The main difference
is that copying doesn’t lose ownership. So, because the original value, is not moved, can still be
used after copy. Rust has implementations of <code>Copy</code> for arrays, which, thus, can yield bad
performances if passed directly to a function, for instance:</p>
<pre><code class="language-rust">fn bar(_: [u32; 1000000]) {}
</code></pre>
<p>Calling <code>bar</code> will make a copy of the array you pass as argument, which is bad news here. However,
I think it’s pretty unlikely that you allocate such an array on your stack. Most non-primitive
types from the standard library don’t have an implementation for <code>Copy</code>, which makes move follow
the move semantics.</p>
<p>Let’s get back to C++, shall we. We’ve established that C++ makes it <em>very easy</em> to write code
which has hidden copies, because of the existence of copy constructors and that value semantics
depends on it. Let’s go on with a problem I have with constructors that is as bad to me.</p>
<h2>The C++ broken initialization design</h2>
<p>Imagine that you want to be able to manipulate a kind of integer. That integer cannot be negative
and it cannot be equal to <code>0</code>. Also, we want it on 32-bit precision. There are several ways to do
that. Let’s enumerate a few of them (non-exhaustive list):</p>
<ol>
<li>Just use the <code>int32_t</code> type from the standard library and every time we try to create or modify
it, forbid to put values which are less or equal to <code>0</code>.</li>
<li>Use the <code>uint32_t</code> type from the standard library and prevent using <code>0</code>. We don’t need to check
for negative values because <code>uint32_t</code> cannot encode them.</li>
<li>Create a wrapper type that prevents from constructing values that violate the invariant (the
invariant being that it cannot be less than <code>1</code>.</li>
</ol>
<p>Obviously, both (1.) and (2.) have a big drawback: you’re going to spend a lot of runtime to check
that the invariant is not violated. Worse, the invariant <em>leaks in the API</em>. They have an API
contract flaw: they don’t say to the programmers they cannot be equal to <code>0</code>. For instance:</p>
<pre><code class="language-cpp">void foo(uint32_t non_zero) {
  // …
}
</code></pre>
<p>Even though the name of the variable is <code>non_zero</code>, what prevents a user from calling <code>foo</code> with
<code>0</code>? Nothing. They can just call <code>foo(0)</code> and the code will still compile. They will then get a
really bad runtime undefined behavior, or maybe nothing for several runs and a sudden bug… or you
will have to handle the invariant at every call, which doesn’t seem like a lot of fun — and it’s
dangerous because you can forget to check it.</p>
<p>Instead, we want this:</p>
<pre><code class="language-cpp">void foo(NonZero non_zero) {
}
</code></pre>
<p>Here, the function will clearly not be callable with a <code>uint32_t</code>. So the provided value <em>must</em> be
statically verified to be non-zero. The <code>NonZero</code> wrapper type must then ensure the invariant is
non-violated. I now have a question: do you think <code>NonZero</code> has to check the invariant every time
it’s used, or we can do better?</p>
<p>C++ states that <code>NonZero</code> should be constructed with a <em>constructor</em>, and my point is that
constructors are broken, because not all values can be constructed via C++ constructors. But they
are not broken the way you think. In theory, and actually, there is a way to use C++ constructors
in sound ways. Try to spend a few minutes and think about it. Can you implement a <code>struct NonZero</code>
with an API exposing <strong>only constructors</strong> to create and initialize non-zero values? There are two
possibilities:</p>
<ul>
<li>Either you, indeed, use a constructor, but because you want to be able to prevent people from
building non-zero values, you will have to <em>fail</em> in your constructor. And failure in constructors
is possible by throwing an <em>exception</em>. I don’t accept that answer because exceptions should
<strong>only be used in exceptional failure situations</strong>, and this is not one of them. I’ll explain
why.</li>
<li>By still allowing construction of the object but putting it in an invalid state. I consider this
a non-answer as well, as you will have to keep track of the invariant every time you want to use
the <code>NonZero</code>, and because it’s very easy to build a <code>NonZero</code> with a violated invariant with your
code compiled correctly: <code>auto foo = NonZero(0);</code>. So, it’s a <em>no</em>.</li>
</ul>
<p>I want to be very very crystal clear here. Exceptions in C++ are heavy. They require unwinding most
of the time when some code throws them. They are needed to handle exceptional errors, such as an
<em>out of memory</em> error, or a dead thread. People have been abusing exceptions for too long. Really.
Stop using exceptions. Calling <code>NonZero(x)</code> with <code>x</code> being <code>0</code> is an error that originate from
several places, because <code>NonZero</code> is a pretty abstract concept. You shouldn’t assume it’s
exceptional that someone tries to build one with <code>0</code>. It’s part of its API: it must fail to
construct if built with <code>0</code>. The API must convey the fact it can handle that kind of error. So
it’s an error that is covered by the scope of the type and its carried invariant.</p>
<p>An example of exceptional error here would be that, when you construct your <code>NonZero</code>, you’re out
of space on the stack. Throwing an error here seems okay to me. The problem, here, is not that we
have a constuctor that fails. The problem is that we used a constructor to encode fallible
construction. That’s the problem. My point here is that constructors must <strong>not</strong> be used for
fallible constructions.</p>
<p>So… what are we left with? If you really want to <em>only</em> use constructors without exceptions,
you have no other choices but to accept to have a dangling invariant. The invariant here is that
the wrapped <code>uint32_t</code> must not be <code>0</code>.</p>
<blockquote>
<p>A small note on invariants: they must be held <em>before</em> and <em>after</em> you call a public function on
your type. We can violate them inside our private code for performance and optimization purposes,
but don’t forget that you must rebalance everything so that the invariant is held when you give
control back to the user. This is especially <strong>hard</strong> to do in C++ because of exception safety, so
a rule of thumb: try to never violate invariants, even in private code.</p>
</blockquote>
<pre><code class="language-cpp">struct NonZero {
  // prevent people from creating default non-zero values; it makes no sense
  NonZero() = delete;

  NonZero(uint32_t value): _wrapped(value) {
    // that would break your application at runtime on debug and do nothing on release… :(
    assert(_wrapped != 0);
  }

  // let C++ automatically implement some stuff for us, which won’t break the invariant
  NonZero(NonZero const &amp;) = default;
  NonZero(NonZero &amp;&amp;) = default;

  NonZero &amp; operator=(NonZero const &amp;) = default;
  NonZero &amp; operator=(NonZero &amp;&amp;) = default;

private:
  // the invariant must be held on this value
  uint32_t _wrapped;
};
</code></pre>
<p>As you can see, having a <code>NonZero</code> with this definition doesn’t mean the value cannot be <code>0</code>.
Example:</p>
<pre><code class="language-cpp">int main() {
  auto nz = NonZero(0); // meh
  return 0;
}
</code></pre>
<p>This code compiles fine and will either abort at runtime on an assert or just silently do something
very wrong on release code.</p>
<p>So what can we do to this situation?</p>
<h2>Construction ≠ constructors</h2>
<p>The thing is that, if you’ve only done C++ in your life, you’re likely to think that the best thing
to do is to give up and use exceptions. After all, they’re pretty nice.</p>
<pre><code class="language-cpp">#include &lt;stdexcept&gt; // you’ll need that for the exception type

struct NonZero {
  // prevent people from creating default non-zero values; it makes no sense
  NonZero() = delete;

  NonZero(uint32_t value): _wrapped(value) {
    if (_wrapped == 0) {
      throw std::invalid_argument(&quot;cannot create a NonZery with 0&quot;);
    }
  }

  // let C++ automatically implement some stuff for us, which won’t break the invariant
  NonZero(NonZero const &amp;) = default;
  NonZero(NonZero &amp;&amp;) = default;

  NonZero &amp; operator=(NonZero const &amp;) = default;
  NonZero &amp; operator=(NonZero &amp;&amp;) = default;

private:
  // the invariant must be held on this value
  uint32_t _wrapped;
};
</code></pre>
<p>Now, when you call <code>NonZero(0)</code>, an exception is thrown. You can catch it with a <code>try</code> / <code>catch</code>
block. But please wait a minute. Several points:</p>
<ul>
<li>Again, exceptions should <strong>not</strong> be used for such errors. I get it you’ve given up on that rule,
but you shouldn’t. Save exceptions for exceptional cases. This is <em>not</em> exceptional. This is
<a href="https://phaazon.net/media/uploads/what_a_story_mark.gif">naaaaaaaaaaht</a>.</li>
<li>When you look at the constructor definition, nothing tells you it can fail. You have to read the
body of the constructor to check for <code>throw</code> — and again, it could quickly get hard because you
might call functions that throw too…</li>
</ul>
<p>That last point is what makes C++ a very hard language what it comes to error handling to me.
Error handling, most of the time, is based on exceptions, which are invisible at the type-level.
I got some remarks stating that, nowadays, modern C++ code bases use the <code>noexcept</code> keyword when
something cannot fail, and the rest of the time, we must assume something can fail. Okay, but
then, fail <em>how</em>? What kind of error? How do I know that? Rely on whether it’s written in a
Doxygen documentation, that, most of the time, doesn’t even exist? Also, just have a look at how
people handle failures in C++. Do you see those <code>try </code> / <code>catch</code> <em>everywhere</em> <code>noexcept</code> is not
annotated? No, because people don’t use <code>noexcept</code> and when they use library, they don’t even
apply that advice to themselves either. The problem with exceptions not being visible in types and
function signatures is that you have to read the code (or the missing Doxygen documentation :)) to
<em>find out</em> what exceptions you need to catch. The other problem is that they can be ignored and
implicitely and automatically propagated upwards in the call stack. There is no way to force a
user to either handle the error or explicitly pass it to their caller.</p>
<p>All that to say: don’t do this. Don’t think constructors are a good idea to construct types in all
possible situations. Failible constructions shouldn’t be done with public constructors and
exceptions. However, it doesn’t mean constructors cannot be used to implement fallible constructions.
That seems tricky, and you’re right. You might wonder:</p>
<blockquote>
<p><em>“Okay okay, you’ve advanced points but… still, how would you do it?”</em></p>
</blockquote>
<p>My implementation — and today, I would only accept that one — to this problem is this: since we
cannot create a value without using a constructor, and since constructors fail to encode fallible
constructions without using exceptions — which I don’t accept either, I’m going to use a private
constructor to build my <code>NonZero</code> and <em>not</em> check the invariant. Then, using a <code>static</code> method, I
can return a more typed object to encode a possible failure, by checking the invariant <em>before</em>
creating the object.</p>
<p>Consider:</p>
<pre><code class="language-cpp">#include &lt;optional&gt;

struct NonZero {
  // prevent people from creating default non-zero values; it makes no sense
  NonZero() = delete;

  // let C++ automatically implement some stuff for us, which won’t break the invariant
  NonZero(NonZero const &amp;) = default;
  NonZero(NonZero &amp;&amp;) = default;

  NonZero &amp; operator=(NonZero const &amp;) = default;
  NonZero &amp; operator=(NonZero &amp;&amp;) = default;

  // static method used to create a NonZero; the only way to create one
  // using the public interface
  std::optional&lt;NonZero&gt; from_u32(uint32_t value) {
    std::optional&lt;NonZero&gt; r;

    if (value != 0) {
      // call to the private ctor; the invariant is already checked in this branch
      r = NonZero(value);
    }

    return r;
  }

private:
  // private constructor that doesn’t check the invariant
  NonZero(uint32_t value): _wrapped(value) {}

  // the invariant must be held on this value
  uint32_t _wrapped;
};
</code></pre>
<p>So let me explain a bit all this code. First, the <code>NonZero(uint32_t)</code> constructor is private. It
has to be private because it doesn’t enforce the invariant. It allows to construct any <code>NonZero</code>.
We need it because C++ requires you to use a constructor to create a value of type <code>NonZero</code>.
Because I don’t want exceptions, I just don’t check the invariant here so that constructor
cannot fail.</p>
<p>The <code>static</code> method called <code>from_u32</code> is the only entry-point to create a value of type
<code>NonZero</code>. As you can see, it returns a <code>std::optional&lt;NonZero&gt;</code>, which means that it can fail.
If you call that function with <code>0</code>, you will get an empty optional value. That function is
implemented by creating an optional value, allocated on the stack with the default constructor —
which makes it empty. Then, we check the invariant and if it’s not violated, we allocate and
initialize the <code>NonZero</code>, and return the whole thing.</p>
<h1>C++ and its bad standard library</h1>
<p>For the rest of the article, consider we add the following method to <code>NonZero</code>:</p>
<pre><code class="language-cpp">  uint32_t value() const {
    return _wrapped;
  }
</code></pre>
<p>If you’ve followed carefully what we’ve been doing here, a call to <code>NonZero::value()</code> will <em>never</em>
return <code>0</code>, because construction statically disallows building such non-zero values. You then don’t
have to check for it!</p>
<p>However, we’re not completely done. We’ve used the <code>std::optional</code> standard type. How are we
supposed to use that type? Looking at the official documentation, we get use the <a href="https://en.cppreference.com/w/cpp/utility/optional/operator*"><code>operator*</code> or
<code>operator-&gt;</code></a> to access the
underlying object. We have the
<a href="https://en.cppreference.com/w/cpp/utility/optional/operator_bool"><code>has_value()</code></a> method to check
whether a value is present.</p>
<pre><code class="language-cpp">auto nz = NonZero::from_u32(0);

if (nz.has_value()) {
  std::cout &lt;&lt; &quot;We have a value! &quot; &lt;&lt; nz-&gt;value() &lt;&lt; std::endl;
}
</code></pre>
<p>That seems nice, right? Well, not really. C++ doesn’t have <em>exhaustive pattern matching</em>, which
makes it impossible to statically ensure you exhaustively check a <code>std::optional</code>. What it means
is that, given the current contract based on <code>has_value()</code> and the dereference operators, we
can still break our program and bring <em>undefined behaviors</em> without having our compiler complain.</p>
<pre><code class="language-cpp">auto nz = NonZero::from_u32(0);

std::cout &lt;&lt; &quot;We have a value… right? &quot; &lt;&lt; nz-&gt;value() &lt;&lt; std::endl;
</code></pre>
<p>That program is perfectly fine and valid. However it’s not. It’s completely wrong, because
<code>std::optional</code>’s API allows developers to access a value that might not be there.</p>
<p>This is the same situation as doing this:</p>
<pre><code class="language-cpp">int * ptr = nullptr;
int x = *ptr;
</code></pre>
<p>Statically fine. Dynamically pretty bad, right? C++’s standard library’s <code>std::optional</code> could
have been done in a much safer way, by, for instance, providing you with some combinators to
work with the underlying type. C++ doesn’t have <em>exhaustive pattern matching</em> but it still has
higher-order functions. We could then have something like:</p>
<pre><code class="language-cpp">auto nz = NonZero::from_u32(0);

nz.maybe(
  // called if no value is present
  []() {
    std::cout &lt;&lt; &quot;No value there.&quot; &lt;&lt; std::endl;
  },
  // called if a value is present
  [](NonZero const &amp;v) {
    std::cout &lt;&lt; &quot;We have a value: &quot; &lt;&lt; v.value() &lt;&lt; std::endl;
  }
);
</code></pre>
<p>For most C++ developers, that code will look like very verbose and wrong, and I agree. Without
pattern matching, it’s very hard to safely and statically use a <code>std::optional</code>.</p>
<h1>How would you do that in Rust?</h1>
<p>In Rust, we have <em>exhaustive static dispatch</em>. <code>std::optional</code> is called <code>Option</code> and Rust
doesn’t have constructors <em>at all</em>.</p>
<p>Values can be constructed by providing <em>all</em> fields, if they are all available (i.e. that’s
always the case in the module the type is defined in; for the rest, you need access to the fields
by marking them all <code>pub</code>).</p>
<p>This is my <code>NonZero</code> wrapper in Rust:</p>
<pre><code class="language-rust">struct NonZero {
  // we don’t make that value pub so no one can directly construct a NonZero
  // outside of our module
  wrapped: u32,
}

impl NonZero {
  // yes, new is nothing special in Rust, so we can use it as a method!
  // Self references the current type we’re adding an impl for (here, NonZero)
  pub new(wrapped: u32) -&gt; Option&lt;Self&gt; {
    if wrapped == 0 {
      None
    } else {
      Some(NonZery { wrapped })
    }
  }
}

fn main() {
  match NonZero::new(0) {
    Some(value) =&gt; println!(&quot;we have a value! {}&quot;, value),
    None =&gt; println!(&quot;we don’t have a value… :(&quot;),
  }
}
</code></pre>
<p>That’s all. The program is statically safe, as the compiler can make sure we’re correctly using
the <code>Option&lt;NonZero&gt;</code>. We could use combinators that do the pattern matching for us. For
instance, imagine that you want the value <code>8080</code> if it’s not correctly constructed, or what it
contains otherwise:</p>
<pre><code class="language-rust">let port = nz.unwrap_or(8080);
</code></pre>
<p>Here, the type of <code>port</code> is <code>NonZero</code>. <code>Option::unwrap_or</code> might be implemented like this:</p>
<pre><code class="language-rust">impl&lt;T&gt; Option&lt;T&gt; {
  pub fn unwrap_or(self, default_value: T) -&gt; T {
    match self {
      Some(v) =&gt; v,
      None =&gt; default_value,
    }
  }
}
</code></pre>
<p>There are a lot of similar combinators, like <code>Option::map</code>, <code>Option::or</code>, <code>Option::unwrap_or_else</code>,
etc. I’ll let you dig by yourselves. However, there is an important point to make here. In C++,
there is the <code>std::optional::value()</code> method that returns the value if it’s present or just throws
an error (which I dislike). In Rust, there is the <code>std::Option::unwrap()</code> method, that returns the
value if it’s present or <em>panics</em> otherwise… which I also dislike. <code>unwrap()</code> can be dangerous in
Rust and I think it should be marked <code>unsafe</code>. But a more detailed article is needed about why
I want it to be marked <code>unsafe</code> and it’s off topic anyways.</p>
<p>So, we’re hitting the end of the article. It might be a lot of information; sorry. To sum up, in
C++:</p>
<ul>
<li>Construction must not be assumed to be done exclusively via constructors.</li>
<li><em>Pure</em> construction (as in infallible) can use constructors if you want.</li>
<li><em>Fallible construction</em> can use:
<ul>
<li>A static function that returns a typed object via, for instance, <code>std::optional</code> or similar,
and enforce invariants / pre-conditions.</li>
<li>A private constructor to build the result value <em>after</em> that invariant and pre-conditions get
checked.</li>
</ul>
</li>
<li>We see that we can make C++ much easier to reason about by following those rules. It will not be
as handy as in Rust where no constructor are needed, but it’s still a good improvement to me.</li>
</ul>
<p>Next time, I’ll bring you on a tour with me around the concept of <em>inclusion polymorphism</em>, which
is most of the time called <em>inheritance</em> in the world of OOP languages. But we’ll still stick to
C++. ;)</p>
<p><a href="https://phaazon.net/media/uploads/the_golden_smile.gif">Keep the vibes!</a></p>
<blockquote>
<p><a href="https://www.reddit.com/r/rust/comments/f8j304/lets_talk_about_c_constructors">Discussion on the article here.</a></p>
</blockquote>
<blockquote>
<p><a href="https://phaazon.net/blog/c++-exceptions">Keep on reading</a>.</p>
</blockquote>


guid: https://strongly-typed-thoughts.net/blog/getting_into_netwireSun, 15 Mar 2015 00:00:00 GMT
title: Getting into netwire
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/getting_into_netwire
feedurl: https://phaazon.net/blog/feed
content:
<h1>Foreword</h1>
<p>Nowadays, there is a cool concept out there in the <em>Functional Programming</em>
world which is called <em>FRP</em>. It stands for <em>Functional Reactive Programming</em>
and is a pretty decent way to make <em>event-driven</em> programs.</p>
<p>The problem with FRP is that, beside <a href="http://en.wikipedia.org/wiki/Functional_reactive_programming">Wikipedia</a>,
<a href="https://wiki.haskell.org/Functional_Reactive_Programming">Haskell.org</a> and a
few other resources, like <a href="http://conal.net/papers/push-pull-frp/">Conal Elliott</a>’s
papers, we lack learning materials. Getting into FRP is really not trivial
and because of the lack of help, you’ll need to be brave to get your feet
wet.</p>
<p>Because I found it <strong>hard</strong> learning it from scratch and because I think
it’s a good thing to pass knowledge by, I decided to write a few about it so
that people can learn via an easier path.</p>
<p>I’ll be talking about <a href="https://hackage.haskell.org/package/netwire">netwire</a>,
which is not the <em>de-facto</em> library to use in Haskell, because, eh… we don’t
have any. However, netwire exposes a lot of very interesting concepts and
helped me to understand more general abstractions. I hope it’ll help you as
well. :)</p>
<h1>The FRP Thing</h1>
<h2>Event-driven programming context</h2>
<p>In traditional event-driven programming codebase, you’d find constructions
such as events polling (when you explicitely retrieve last occurring events),
callbacks and <em>event handlers</em>. <a href="http://www.glfw.org/">GLFW</a> is a very famous
example of callback uses for event-driven programming. Such functions as
<a href="http://www.glfw.org/docs/latest/group__window.html#gaade9264e79fae52bdb78e2df11ee8d6a">glfwSetWindowCloseCallback</a>
require you to pass a callback that will be called when the event occurs. While
that way to go seems nice, it’s actually error-prone and ill-formed design:</p>
<ul>
<li>you eventually end up with <a href="http://en.wikipedia.org/wiki/Spaghetti_code">spaghetti code</a></li>
<li>debugging callbacks is a true nightmare as the codebase grows</li>
<li>because of the first point, the code doesn’t compose – or in very minimal
ways – and is almost impossible to test against</li>
<li>you introduce side-effects that might introduce nasty bugs difficult to
figure out</li>
<li>debugging is like hitting your head on a tree log</li>
</ul>
<p><img src="http://phaazon.net/pub/spaghetti_code.jpg" alt="" /></p>
<p>However, it’s not black or white. Callbacks are mandatory. They’re useful, and
we’ll see that later on.</p>
<h2>What FRP truely is?</h2>
<p>FRP is a new way to look at event-driven programming. Instead of representing
reaction through callbacks, we consume events over time. Instead of building a
callback that will be passed as reaction to the <code>setPushButtonCallback</code>
function, we consume and transform events over time. The main idea of FRP could
be summed up with the following concepts:</p>
<ul>
<li><em>behaviors</em>: a behavior is a value that reacts to time</li>
<li><em>events</em>: events are just values that have occurrences in time</li>
<li><em>switching</em>: the act of changing of behavior</li>
</ul>
<h3>Behaviors</h3>
<p><a href="http://en.wikipedia.org/wiki/Behavior">According to Wikipedia</a>, <em>a behavior is the
range of actions and mannerisms made by individuals, organisms, systems, or artificial
entities in conjunction with themselves or their environment, which includes the other
systems or organisms around as well as the (inanimate) physical environment</em>. If we try
to apply that to a simple version of FRP that only takes into account the time as
external stimulus, a behavior isany kind of value that consumes time. What’s that?
Well…</p>
<pre><code>newtype Behavior a = Behavior { stepBehavior :: Double -&gt; a }
</code></pre>
<p>A behavior is a simple function from time (<code>Double</code>) to a given value. Let’s take an
example. Imagine you want to represent a cube rotating around the <em>X axis</em>. You
can represent the actual rotation with a <code>Behavior Rotation</code>, because the angle of
rotation is linked to the time:</p>
<p><img src="http://phaazon.net/pub/human_behavior.jpg" alt="" /></p>
<pre><code>rotationAngle :: Behavior Rotation
rotationAngle = Behavior $ \t -&gt; rotate xAxis t
</code></pre>
<p>Pretty simple, see?! However, it’d would more convenient if we could chose the type
of time. We don’t really know what the time will be in the final application. It
could be the current UTC time, it could be an integral time (think of a stepped
discrete simulation), it could be the monotonic time, the system time, something
that we don’t even know. So let’s make our <code>Behavior</code> type more robust:</p>
<pre><code>newtype Behavior t a = Behavior { stepBehavior :: t -&gt; a }
</code></pre>
<p>Simple change, but nice improvement.</p>
<p>That is the typical way to picture a <em>behavior</em>. However, we’ll see later that
the implementation is way different that such a naive one. Keep on reading.</p>
<h3>Events</h3>
<p>An event is <em>something happening at some time</em>. Applied to FRP, an event is a pair
of time – giving the time of occurrence – and a carried value:</p>
<pre><code>newtype Event t a = Event { getEvent :: (t,a) }
</code></pre>
<p>For instance, we could create an event that yields a rotation of 90° around X
at 35 seconds:</p>
<pre><code>rotate90XAt35s :: Event Float Rotation
rotate90XAt35s = Event (35,rotate xAxis $ 90 * pi / 180)
</code></pre>
<p>Once again, that’s the naive way to look at events. Keep in mind that events have
time occurrences and carry values.</p>
<h3>Behavior switch</h3>
<p>You switch your behavior every time. Currently, you’re reading this paper, but you
may go grab some food, go to bed, go to school or whatever you like afterwards.
You’re already used to behavior switching because that’s what we do every day in
a lifetime.</p>
<p>However, applying that to FRP is another thing. The idea is to express this:</p>
<blockquote>
<p><em>“Given a first behavior, I’ll switch to another behavior when a given event
occurs.”</em></p>
</blockquote>
<p>This is how we express that in FRP:</p>
<pre><code>switch :: Behavior t a -&gt; Event t (Behavior t a) -&gt; Behavior t a
</code></pre>
<p>Let me decrypt <code>switch</code> for you.</p>
<p>The first parameter, a <code>Behavior t a</code>, is the
initial behavior. For instance, currently, you’re reading. That could be the
first behavior you’d pass to <code>switch</code>.</p>
<p>The second parameter, an <code>Event t (Behavior t a)</code>, is an event that yields a
new <code>Behavior t a</code>. Do you start to get it? No? Well then:</p>
<pre><code>switch reading finished
</code></pre>
<p><code>reading</code> is the initial behavior, and <code>finished</code> is an event that occurs when
you’re done reading. <code>switch reading finished</code> is then a behavior that equals
to <code>reading</code> until <code>finished</code> happens. When it does, <code>switch reading finished</code>
extracts the behavior from the event, and uses it instead.</p>
<p>I tend to think <code>switch</code> is a bad name, and I like naming it <code>until</code>:</p>
<pre><code>reading `until` finished
</code></pre>
<p>Nicer isn’t it?! :)</p>
<h3>Stepping</h3>
<p>Stepping is the act of passing the input – i.e. the time <code>t</code> in our case – down
to the <code>Behavior t a</code> and extract the resulting <code>a</code> value. Behaviors are
commonly connected to each other and form a <em>reactive network</em>.</p>
<p>That operation is also often refered to as <em>reactimation</em> in certain
implementations, but is more complex than just stepping the world. You don’t
have to focus on that yet, just keep in mind the <code>reactimate</code> function. You
might come across it at some time.</p>
<h1>Before going on…</h1>
<p>Everything you read in that paper until now was just pop-science so that you
understand the main idea of what FRP is. The next part will cover a more
decent and real-world implementation and use of FRP, especially efficient
implementations.</p>
<h1>Let’s build a FRP library!</h1>
<p>The first common error a lot of programmers make is trying to write
algorithms or libraries to solve a problem they don’t even know. Let’s then
start with an example so that we can figure out the problem.</p>
<h2>Initial program</h2>
<h3>Setting up</h3>
<p>Let’s say we want to be able to control a camera with a keyboard:</p>
<ul>
<li><code>W</code> would go forward</li>
<li><code>S</code> would go backward</li>
<li><code>A</code> would left strafe</li>
<li><code>D</code> would right strafe</li>
<li><code>R</code> would go up</li>
<li><code>F</code> would go down</li>
</ul>
<p><img src="http://phaazon.net/pub/camera_drawing.gif" alt="" /></p>
<p>Let’s write the <code>Input</code> type:</p>
<pre><code class="language-haskell">data Input
  = W
  | S
  | A
  | D
  | R
  | F
  | Quit
    deriving (Eq,Read,Show)
</code></pre>
<p>Straight-forward. We also have a function that polls events from <code>IO</code>:</p>
<pre><code class="language-haskell">pollEvents :: IO [Input]
pollEvents = fmap treatLine getLine
  where
    treatLine = concatMap (map fst . reads) . words
</code></pre>
<p>We use <code>[Input]</code> because we could have several events at the same time
(imagine two pressed keys). The function is using dumb implementation
in order to abstract over event polling. In your program, you won’t use
<code>getLine</code> but a function from <a href="https://hackage.haskell.org/package/sdl2">SDL</a>
or similar.</p>
<p>And the camera:</p>
<pre><code class="language-haskell">newtype Camera = Camera { _cameraPosition :: V3 Float } deriving (Eq,Read,Show)

makeLenses ''Camera
</code></pre>
<p><code>V3</code> is a type from <a href="https://hackage.haskell.org/package/linear">linear</a>.
You’ll need that lib then, and <code>import Linear.V3</code> to make the <code>Camera</code> compile.
You’ll also need <a href="https://hackage.haskell.org/package/lens">lens</a> and the GHC
<code>TemplateHaskell</code> extension enabled as well as <code>import Control.Lens</code>.</p>
<p>Ok, let’s react to events!</p>
<h3>First attempt: the regular and naive one</h3>
<p>The idea is to use some kind of state we’ll change on an event. In our case the
state is pretty simple:</p>
<pre><code class="language-haskell">data AppSt = AppSt {
    _appCamera :: Camera
  } deriving (Eq,Read,Show)
  
makeLenses ''AppSt

updateAppSt :: AppSt -&gt; Input -&gt; Maybe AppSt
updateAppSt appst input = case input of
  W -&gt; Just $ appst &amp; appCamera . cameraPosition . _z -~ 0.1
  S -&gt; Just $ appst &amp; appCamera . cameraPosition . _z +~ 0.1
  A -&gt; Just $ appst &amp; appCamera . cameraPosition . _x -~ 0.1
  D -&gt; Just $ appst &amp; appCamera . cameraPosition . _x +~ 0.1
  F -&gt; Just $ appst &amp; appCamera . cameraPosition . _y -~ 0.1
  R -&gt; Just $ appst &amp; appCamera . cameraPosition . _y +~ 0.1
  Quit -&gt; Nothing
</code></pre>
<p>A lot of boilerplate on <code>updateAppSt</code>, but that doesn’t matter that much.
The idea is to modify the application state and just return it for all
inputs but <code>Quit</code>, in which case we return <code>Nothing</code> to express the wish
to quit the application.</p>
<p>I’ve been using that idea for a while. It’s simple, nice and neat, because
we don’t spread <code>IO</code> actions in our program logic, which remains then pure.
That’s a very good way of doing it, and in most cases, it’ll even be
sufficient. However, that idea suffers from a serious issue: <em>it doesn’t
scale</em>.</p>
<p>Who has only one camera? No one. You have a camera – maybe more than just
one, lights, objects, terrains, AI, sounds, assets, maps and so on and so
forth. Our little <code>AppSt</code> type would explode as we add objects. That
<em>doesn’t scale at all</em>. You could, though, go on and add all your objects
in your <code>AppSt</code> – I did that once, it was a pretty harsh experience.</p>
<p>Furthermore, imagine you want to add a new behavior to the camera, like
being able to handle the mouse cursor move – <code>Input</code> being augmented, of
course. You’d need to change / add lines in our <code>updateAppSt</code> function.
Imagine how messy <code>updateAppSt</code> would be… That would, basically, gather
all reactions into a single function. Not neat.</p>
<h2>Adding FRP</h2>
<p>FRP enables us to use our reactive values as if they are regular values.
You can add reactive values, you can substract them, combine them in any
way you want. The semantics of your values should be true for the
reactive values.</p>
<p>Typically, with FRP, you don’t have event handlers anymore. The codebase
can then grow sanely without having to accumulate big states every now
and then. FRP applications scale and compose pretty well.</p>
<p>Let’s start with a simple FRP implementation for our example.</p>
<h3>Naive FRP implementation</h3>
<p>Let’s start with the behavior:</p>
<pre><code>newtype Behavior t a = Behavior { stepBehavior :: t -&gt; a }
</code></pre>
<p>How could we implement our camera’s behavior with that? We actually
can’t since we don’t have any ways to pass events.</p>
<blockquote>
<p><em>“I guess we could build a <code>Behavior t Camera</code> by passing our <code>Input</code>
to the initial function?”</em></p>
</blockquote>
<p>Something like this?</p>
<pre><code class="language-haskell">camera inputs = Behavior $ \_ -&gt; -- implement the behavior with inputs
</code></pre>
<p>That could be a way to go, yeah. However, how would you implement
switching with that? Remember the type of <code>until</code>:</p>
<pre><code>until :: Behavior t a -&gt; Event (Behavior t a) -&gt; Behavior t a
</code></pre>
<p><code>camera</code> is not a behavior, it’s a function from events to a
behavior. You have to apply the events on <code>camera</code> in order to get its
behavior. Once you’ve done that, you cannot pass events to the next
behavior. What a pity. That is more a configured behavior than a
behavior consuming inputs / events.</p>
<p>With the current <code>Behavior t a</code> implementation, a behavior network is
reduced to a function <code>t -&gt; a</code>, basically. Then, the only stimulus you
got from outside is… <em>time</em>. We lack something.</p>
<h3>Arrowized behaviors</h3>
<blockquote>
<p><em>“A way to forward events?”</em></p>
</blockquote>
<p><img src="https://www.haskell.org/arrows/first.png" alt="" /></p>
<p>Yes! But more mainly, a way to extend our <code>Behavior t a</code> with inputs!
Don’t get me wrong, we are not talking about a reactive <em>value</em> here.
We are talking about a reactive <em>relationship</em>:</p>
<pre><code>newtype Behavior t a b = Behavior { stepBehavior :: t -&gt; a -&gt; b }
</code></pre>
<p>That’s way better! Our new behavior represents a relationship between
two reactive objects. The <code>b</code> is our old <code>a</code>, and the new <code>a</code> is the
input! Let’s see what we can do with that.</p>
<pre><code class="language-haskell">camera :: Behavior t [Input] Camera
camera = Behavior (const treatEvents)
  where
    treatEvents events
      | W `elem` events = Camera $ V3 0 0 (-0.1)
      | S `elem` events = Camera $ V3 0 0 0.1
      | otherwise = Camera $ V3 0 0 0
</code></pre>
<p>That is not exactly what we intented to express. Here, if we push the
<code>W</code> button, we just put the camera in <code>V3 0 0 (-0.1)</code>, while we’d like
to move it forward. That is due to the fact we need switching.</p>
<p>The idea is the following: the initial behavior is idleing. We just
don’t do anything. Then, we switch to a given behavior when a given
event occurs. We’ll need recursion so that we can <em>ping-pong</em> between
behaviors. Let’s write the <code>idleing</code> behavior:</p>
<pre><code class="language-haskell">idleing :: Behavior t ([Input],Camera) Camera
idleing = Behavior (const snd)
</code></pre>
<p>That behavior requires as input our <code>Input</code> events list and a
<code>Camera</code> and simply returns the <code>Camera</code>. Pretty nice.</p>
<p>How do we switch then? We need <code>Event</code>. Consider this:</p>
<pre><code>newtype Event t a = Event { getEvent :: (t,a) }
</code></pre>
<p>In order to switch, we need <code>a</code> to be a behavior. In the first place,
we’ll create several <code>Event t [Input]</code> and pass them as input to the
behavior network. How could we change the <code>[Input]</code> to something more 
interesting? Simple: <a href="https://wiki.haskell.org/Functor">Functor</a>!</p>
<pre><code class="language-haskell">instance Functor (Event t) where
  fmap f (Event e) = Event (fmap f e)
</code></pre>
<blockquote>
<p><strong>Note</strong>: because of <code>Event t a</code> being a <code>newtype</code>, you should
rather use the GHC <code>GeneralizedNewtypeDeriving</code> extension to
automatically let GHC infer the instance for you.</p>
</blockquote>
<pre><code>newtype Event t a = Event { getEvent :: (t,a) } deriving (Functor)
</code></pre>
<p>Then, we can use the <code>Functor</code> instance to change the type of the
carried value of the event. That’s great because we don’t change the
occurrence time, only the carried value. Transforming events is really
important in FRP. We can then transform the <code>[Input]</code> into a single
behavior:</p>
<pre><code class="language-haskell">inputToBehavior i = case i of
  W -&gt; Behavior $ \_ (_,cam) -&gt; cam &amp; cameraPosition . _z -~ 0.1
  S -&gt; Behavior $ \_ (_,cam) -&gt; cam &amp; cameraPosition . _z +~ 0.1
  A -&gt; -- and so on
  D -&gt; -- and so forth
  F -&gt; -- ;)
  R -&gt; -- ...
  _ -&gt; Behavior $ \_ (_,cam) -&gt; cam
</code></pre>
<p>Pretty simple, see? When we push <code>W</code>, we go forward forever. We could
have implemented the function above with another <code>until</code> call in order
to go back to <code>idleing</code>, making some kind of behavior loop.</p>
<p>However, switching is fairly poorly implemented here. It’s not very
efficient, and requires a ton of boilerplate.</p>
<h2>Auto</h2>
<p>There is a very cool type out there called <code>Auto</code>, used to implement
<a href="http://en.wikipedia.org/wiki/Automaton">automatons</a>.</p>
<pre><code>newtype Auto a b = Auto { runAuto :: a -&gt; (b,Auto a b) }
</code></pre>
<p>An <code>Auto a b</code> is a function-like structure. It has an input and an
output. The difference with a regular function is the fact it also has
a secondary output, which is another <code>Auto a b</code>. That is, it’s the next
automaton to be used.</p>
<p><code>Auto a b</code> wraps pretty cool concepts, such as <em>locally defined states</em>.
It’s also a great ally when implementing switching in a FRP system,
because we can easily state that <code>Behavior</code> ≃ <code>Auto</code>. A <code>Behavior</code> is
a function from the environment state to the next reactive value, and
has also another output representing what to do “next”.</p>
<p>Let’s then change our <code>Behavior</code> type to make it look like a bit more
like <code>Auto</code>:</p>
<pre><code>newtype Behavior t a b = Behavior { stepBehavior :: t -&gt; a -&gt; (b,Behavior t a b) }
</code></pre>
<p>Yeah, that’s it! That’s a pretty good start!</p>
<h2>Useful abstractions</h2>
<p>Before going on, I’d like to introduce those scary abstractions you are
afraid of. Because they’re actually not. They’re <strong>all simple</strong>. At least
for Haskell purposes.</p>
<p><strong>Note</strong>: I do know we could simply use the GeneralizedNewtypeDeriving`
extension but I want to detail all the implementation, so we’ll see
how to implement all the nice abstractions.</p>
<h3>Arrow</h3>
<p><a href="https://www.haskell.org/arrows">Arrows</a> are a generalization of functions
along the axis of computation. A computation has inputs and outputs. So
does a behavior.</p>
<p>Although arrows are not really used in Haskell, they’re
ultra simple (themselves and the common combinators built over the
abstraction) and useful in some cases.</p>
<p>In order to implement arrows, we need to provide code for both the <code>arr</code>
function, which type is <code>arr :: (Arrow a) =&gt; (b -&gt; c) -&gt; a b c</code> and
<code>first</code>, which type is <code>first :: (Arrow a) =&gt; a b c -&gt; a (b,d) (c,d)</code>.
<code>arr</code> is used to lift a common function into the arrowized version, and
<code>first</code> takes an arrow which takes a value as input and exposes an arrow
that takes a pair as input, applying the given function on the <em>first</em>
value of the pair. Let’s implement that:</p>
<pre><code class="language-haskell">instance Arrow (Behavior t) where
  arr f = fix $ \r -&gt; Behavior $ \t a -&gt; (f a,r)
  first f = Behavior $ \t (b,d) -&gt;
    let (c,fn) = stepBehavior f t b
    in ((c,d),first fn)
</code></pre>
<h3>Category</h3>
<p>A <a href="http://en.wikipedia.org/wiki/Category_%28mathematics%29">category</a>
basically exposes two concepts: composition and identity. In our case,
the identity represents a constant behavior in time and the composition
composes two behaviors in time. Let’s implement <code>Category</code> by providing
implementation for both <code>id</code> and <code>(.)</code>:</p>
<pre><code class="language-haskell">instance Category (Behavior t) where
  id = arr id
  x . y = Behavior $ \t a -&gt;
    let (yr,yn) = stepBehavior y t a
        (xr,xn) = stepBehavior x t yr
    in (xr,xn . yn)
</code></pre>
<p><strong>Note</strong>: because of <code>Prelude</code> exporting specialized implementation
of <code>id</code> and <code>(.)</code> – the function ones – you should hide them in order
to implement <code>Category</code>:</p>
<pre><code>import Prelude hiding ( (.), id )
</code></pre>
<h3>Semigroup</h3>
<p>A <a href="http://en.wikipedia.org/wiki/Semigroup">semigroup</a> is a pretty cool
algebraic structure used in Haskell to represent “anything that
associates”. It exposes an associative binary function over a set.
In the case of behaviors, if two behaviors output semigroup values, we
can associates the behaviors to build a single one.</p>
<p>A <code>Semigroup</code> is implemented via a single typeclass method, <code>(&lt;&gt;)</code>.
Let’s do that for behaviors:</p>
<pre><code class="language-haskell">instance (Semigroup b) =&gt; Semigroup (Behavior t a b) where
  x &lt;&gt; y = Behavior $ \t a -&gt;
    let (xr,xn) = stepBehavior x t a
        (yr,yn) = stepBehavior y t a
    in (xr &lt;&gt; yr,xn &lt;&gt; yn)
</code></pre>
<p>Simple and neat.</p>
<h3>Functor</h3>
<p>You might already know that one since I talked about it a few lines ago,
but let’s write the instance for our <code>Behavior</code>:</p>
<pre><code class="language-haskell">instance Functor (Behavior t a) where
  fmap f b = Behavior $ \t a -&gt;
    let (br,bn) = stepBehavior b t a
    in (f br,fmap f bn)
</code></pre>
<p>Pretty cool eh?</p>
<h3>Applicative</h3>
<p>A very known one too. Let’s see how we could implement <code>Applicative</code>:</p>
<pre><code class="language-haskell">instance Applicative (Behavior t a) where
  pure = arr . const
  f &lt;*&gt; x = Behavior $ \t a -&gt;
    let (fr,fn) = stepBehavior f t a
        (xr,xn) = stepBehavior x t a
    in (fr xr,fn &lt;*&gt; xn)
</code></pre>
<h3>Profunctor</h3>
<p>This one is special. You don’t have to know what a profunctor is, but
eh, you should, because profunctors are pretty simple to use in Haskell,
and are very useful. I won’t explain what they are – you should have a look
at <a href="https://www.fpcomplete.com/school/to-infinity-and-beyond/pick-of-the-week/profunctors">this article</a>
for further details.</p>
<p>If you do know them, here’s the implementation for <code>dimap</code>:</p>
<pre><code class="language-haskell">instance Profunctor (Behavior t) where
  dimap l r x = Behavior $ \t a -&gt;
    let (xr,xn) = stepBehavior x t (l a)
    in (r xr,dimap l r xn)
</code></pre>
<h2>Inhibition</h2>
<h3>Bare concept</h3>
<p>Behaviors consume environment state and have outputs. However, they
sometimes just don’t. They don’t output anything. That could be the
case for a behavior that only emits during a certain period of time.
It could also be the case for a signal function that’s defined on a
given interval: what should we output for values that lie outside?</p>
<p>Such a scenario is called <em>inhibition</em>. There’re several solutions
to implement inhibition. The simplest and most famous one is by
using <code>Maybe</code> as a wrapper over the output. Like the following:</p>
<pre><code>Behavior t a (Maybe b)
</code></pre>
<p>If <code>(Maybe b)</code> is <code>Nothing</code>, the output is undefined, then the
behavior inhibits.</p>
<p>However, using a bare <code>Maybe</code> exposes the user directly
to inhibition. There’s another way to do that:</p>
<pre><code>newtype Behavior t a b = Behavior { stepBehavior :: t -&gt; a -&gt; Maybe (b,Behavior t a b) }
</code></pre>
<p>Here we are. We have behaviors that can inhibit. If a behavior doesn’t
inhibit, it returns <code>Just (output,nextBehavior)</code>, otherwise it
outputs <code>Nothing</code> and inhibits forever.</p>
<blockquote>
<p><strong>Exercise</strong>: try to reimplement all the above abstractions with
the new type of <code>Behavior</code>.</p>
</blockquote>
<p>We can add a bunch of other interesting functions:</p>
<h3>Inhibition-related combinators</h3>
<pre><code class="language-haskell">dead :: Behavior t a b
dead = Behavior $ \_ _ -&gt; Nothing

one :: b -&gt; Behavior t a b
one x = Behavior $ \_ _ -&gt; Just (x,dead)
</code></pre>
<p><code>dead</code> is a behavior that inhibits forever. That is, it doesn’t
produce any value at any time.</p>
<p><code>one x</code> produces <code>x</code> once, and then inhibits forever. That’s a nice
combinator to use when you want to <em>pulse</em> values in time. We’ll
see later that it’s very useful to represent discrete events, like
key presses or mouse motion.</p>
<p>However, inhibiting can be useful. For instance, we can implement
a new kind of behavior switching using inhibition. Let’s try to
implement a function that takes two behaviors and switches to the
latter when the former starts inhibiting:</p>
<pre><code class="language-haskell">revive :: Behavior t a b -&gt; Behavior t a b -&gt; Behavior t a b
revive x y = Behavior $ \t a -&gt; case stepBehavior x t a of
  Just (xr,xn) -&gt; return (xr,revive xn y)
  Nothing -&gt; stepBehavior y t a

(~&gt;) :: Behavior t a b -&gt; Behavior t a b -&gt; Behavior t a b
(~&gt;) = revive
</code></pre>
<p><code>(~&gt;)</code> is a handy alias to <code>revive</code>. Then, <code>a ~&gt; b</code> is a behavior
that is <code>a</code> until it inhibits, afterwhile it’s <code>b</code>. Simple, and
useful.</p>
<p>In <em>netwire</em>, <code>revive</code> – or <code>(~&gt;)</code> – is <code>(--&gt;)</code>. There’s
also an operator that does the opposite thing: <code>(&gt;--)</code>.
<code>a &gt;-- b</code> is <code>a</code> until <code>b</code> starts producing – i.e. until <code>b</code>
doesn’t inhibit anymore.</p>
<blockquote>
<p><strong>Exercise</strong>: write the implementatof of <code>(&gt;~)</code>, our version
for netwire’s <code>(&gt;--)</code>.</p>
</blockquote>
<p><img src="http://phaazon.net/pub/wire.jpg" alt="" /></p>
<h2>Behaviors revisited</h2>
<p>Now you have a better idea of how you could implement a
behavior, let’s talk about netwire’s one.</p>
<p>netwire’s behavior type is called <code>Wire</code>. It’s actually:</p>
<pre><code>Wire s e m a b
</code></pre>
<p><code>s</code> is the <em>session</em> time – it’s basically a type that’s used
to extract time. <code>e</code> is the inhibition value. <code>m</code> is a inner
monad – yes, you can use monadic code within netwire, which is
not really useful actually, except for <code>Reader</code>, I guess. And
<code>a</code> and <code>b</code> are respectively inputs and outputs.</p>
<blockquote>
<p><em>“What is that inhibition type?”</em></p>
</blockquote>
<p>Yeah, netwire doesn’t use <code>Maybe</code> for inhibition. Picture 
<code>Wire</code> as:</p>
<pre><code>newtype Wire s e m a b = Wire { stepWire :: s -&gt; a -&gt; m (Either e (b,Wire s e m a b)) }
</code></pre>
<p>Instead of using <code>Maybe (b,Wire s e m a b)</code>, it uses <code>Either</code>.
Some functions require <code>e</code> to be a <code>Monoid</code>. I guess netwire uses
that to accumulate during inhibition. I don’t see decent use
cases of such a feature, but it’s there. I tend to use this kind
of wire in all my uses of netwire:</p>
<pre><code class="language-haskell">Wire s () Identity a b -- I think this is the most common type of wire
</code></pre>
<p>Keep in mind that although you can set <code>m</code> to <code>IO</code>, it’s not
what netwire – and FRP – was designed for.</p>
<h2>Events</h2>
<p>What about events? Well, netwire exposes events as a home-made
<code>Maybe</code>:</p>
<pre><code class="language-haskell">data Event a
  = Event a
  | NoEvent
    deriving (Eq,Show)

instance Functor Event where
  fmap f e = case e of
    Event a -&gt; Event (f a)
    NoEvent -&gt; NoEvent
</code></pre>
<p>That’s actually enough, because we can attach <code>Event a</code> to time
occurences with <code>Behavior t b a</code>. You’ll find every now and then
functions using <code>Wire s e m (Event a) b</code>, for instance. You should
get used to that as you write toy examples, and real-world ones,
of course.</p>
<h1>In the end</h1>
<p>What a trek… As you can see, we were able to approach netwire’s
implementation understanding pretty closely. There are a few concepts
I haven’t covered – like intrinsic switches, continuable switches,
deferred switches… – but I don’t pretend having a comprehensive
FRP article. You’ll have to dig in a bit more ;)</p>
<p>I’ll write another article about FRP and netwire to implement the
camera example with netwire so that you can have a concrete example.</p>


guid: https://strongly-typed-thoughts.net/blog/shades-edslSat, 30 Jul 2022 13:30:00 GMT
title: A Rust shading language EDSL
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/shades-edsl
feedurl: https://phaazon.net/blog/feed
content:
<p>This blog article is about a crate I have been working on for a while now: <a href="https://crates.io/crates/shades">shades</a>. That library is a <em>shading
library</em>, that is, it provides you with types, traits and functions to create <a href="https://en.wikipedia.org/wiki/Shader">shaders</a>. A shader is just a piece of
<em>code</em> that typically runs on your GPU, and is written in a language such as <a href="https://www.khronos.org/opengl/wiki/OpenGL_Shading_Language">GLSL</a> — or anything that can target
<a href="https://www.khronos.org/spir">SPIR-V</a> these days.</p>
<h1>Context &amp; problem</h1>
<p>The way it typically works is by writing the GLSL code in a hard-coded string, or a file that is dynamically loaded at
runtime by your application (some people even pre-compile GLSL into a binary form, but it doesn’t change the fact that
the binary has to be brought to the GPU at runtime). The overall implication of all this is that the shader is
<em>represented</em> via an opaque <code>String</code> (or binary like <code>Vec&lt;u8&gt;</code>) in your application.</p>
<p>A couple of examples of interfaces to handle shaders, using <a href="https://crates.io/crates/wgpu">wgpu</a> and a crate of mine, <a href="https://crates.io/crates/luminance">luminance</a>:</p>
<ul>
<li><a href="https://docs.rs/wgpu/0.12.0/wgpu/enum.ShaderSource.html"><code>wgpu::ShaderSource</code></a></li>
<li><a href="https://docs.rs/luminance/0.47.0/luminance/shader/struct.ProgramBuilder.html#method.from_strings"><code>luminance::shader::ProgramBuilder::from_strings</code></a></li>
</ul>
<p>As you can see, dealing with shader sources implies a string interface, because that’s basically code that needs to be
compiled by your GPU driver (done at runtime). That has several issues:</p>
<ul>
<li>There is no way to <em>inspect</em> the content of the shader to use its inputs, outputs, environment variables, etc. That
leads to a lot of code duplication, which is just <code>String</code> duplication all over the place.</li>
<li>Your application might compile and run, you are not sure that your shaders are correctly written, so you have to write
a runtime checker for your shader, that needs to go through all the shaders in use in your application. Tedious, and
error-prone.</li>
<li>You have to learn a shading language. Most of the time, <a href="https://www.khronos.org/opengl/wiki/OpenGL_Shading_Language">GLSL</a> should be enough, but <a href="https://www.khronos.org/opengl/wiki/OpenGL_Shading_Language">GLSL</a> is old, and not very fun
to work with.</li>
<li>You will have to write the same code over and over and over. For instance, in <a href="https://www.khronos.org/opengl/wiki/OpenGL_Shading_Language">GLSL</a>, there is no definition of π,
so you will have to write its definition whenever you need it. That will quickly lead you to write some combinator
code to inject sub strings containing such definitions and it will be a nightmare.</li>
<li>It’s very weakly typed. A <code>String</code> doesn’t allow you to be sure that shader is compatible with the graphics pipeline.
Being the author of <a href="https://crates.io/crates/luminance">luminance</a>, that is a massive flaw to me. As you might know, the typing in <a href="https://crates.io/crates/luminance">luminance</a> is pretty
advanced (phantom typing, type families, type states, etc.), so being stuck with <code>String</code> is not a good situation.</li>
</ul>
<h1>Introducing <code>shades</code></h1>
<p>I introduced <a href="https://crates.io/crates/shades">shades</a> a couple of years ago without going public about it. The reason for that was that it was not
completely ready for people to use. Of course you could have given a try (and fundamentally, it hasn’t changed much),
but the syntax was going to drive people off right away. The idea of <a href="https://crates.io/crates/shades">shades</a> is to write shaders directly in Rust.
Instead of using <code>String</code> to represent a shader stage, you use <code>Stage&lt;Inputs, Outputs, Environment&gt;</code>. That gives you a
first visible advantage: typing. In <a href="https://crates.io/crates/shades">shades</a>, you don’t have to declare your inputs, outputs nor environment. For
instance, compare the following GLSL code with its <a href="https://crates.io/crates/shades">shades</a> counterpart:</p>
<pre><code class="language-glsl">uniform float time;
uniform vec2 resolution;
</code></pre>
<p>Whenever a shader stage has to use <code>time</code> or <code>resolution</code>, they will have to declare those two lines at the top of their
sources. With <a href="https://crates.io/crates/shades">shades</a>:</p>
<pre><code class="language-rust">#[derive(Debug)]
struct Env {
  time: f32,
  resolution: V2&lt;f32&gt;,
}

#[derive(Debug)]
struct EnvExpr {
  time: Expr&lt;f32&gt;,
  resolution: Expr&lt;V2&lt;f32&gt;&gt;,
}

impl Environment for Env {
  type Env = EnvExpr;

  fn env() -&gt; Self::Env {
    EnvExpr {
      time: Expr::new_env(0),
      resolution: Expr::new_env(1),
    }
  }

  fn env_set() -&gt; Vec&lt;(u16, Type)&gt; {
    vec![
      (0, &lt;f32 as ToType&gt;::to_type()),
      (1, &lt;V2&lt;f32&gt; as ToType&gt;::to_type()),
    ]
  }
}
</code></pre>
<p>When using a <code>Stage&lt;_, _, Env&gt;</code>, <code>time</code> and <code>resolution</code> will automatically be available, whenever you use them or not.
This is even more interesting with procedural macro support in <a href="https://crates.io/crates/luminance">luminance</a> for instance:</p>
<pre><code class="language-rust">// the Environment and Semantics derive macro will generate all the above code for you, as well as the type family
#[derive(Debug, Environment, Semantics)]
struct Env {
  time: f32,
  resolution: V2&lt;f32&gt;,
}
</code></pre>
<p>So yes, <a href="https://crates.io/crates/shades">shades</a> is a bit more verbose at the Rust level, but I think it’s worth it because shaders are checked at
compile-time (via <code>rustc</code>) and not at runtime anymore. Typing also allows a better composition and reusability.</p>
<p>Just to give you an idea, this is an old and outdated example taken from the repository to show you what it used to be:</p>
<pre><code class="language-rust">let vertex_shader = StageBuilder::new_vertex_shader(
  |mut shader: StageBuilder&lt;MyVertex, (), ()&gt;, input, output| {
    let increment = shader.fun(|_: &amp;mut Scope&lt;Expr&lt;f32&gt;&gt;, a: Expr&lt;f32&gt;| a + lit!(1.));

    shader.fun(|_: &amp;mut Scope&lt;()&gt;, _: Expr&lt;[[V2&lt;f32&gt;; 2]; 15]&gt;| ());

    shader.main_fun(|s: &amp;mut Scope&lt;()&gt;| {
      let x = s.var(1.);
      let _ = s.var([1, 2]);
      s.set(output.clip_distance.at(0), increment(x.clone()));
      s.set(
        &amp;output.position,
        vec4!(input.pos, 1.) * lit![0., 0.1, 1., -1.],
      );

      s.loop_while(true, |s| {
        s.when(x.clone().eq(1.), |s| {
          s.loop_break();
          s.abort();
        });
      });
    })
  },
);

let output = shades::writer::glsl::write_shader_to_str(&amp;vertex_shader).unwrap();
</code></pre>
<p>That snippet doesn’t do anything really logical; it’s just there to show you how things look like.</p>
<p>I wrote a couple of things with <a href="https://crates.io/crates/shades">shades</a>, and even though I like it, I have to admit that the Rust API is pretty hard to
read, use and understand. I needed something better.</p>
<h1>The new age of <code>shades</code>: <code>shades-edsl</code></h1>
<p>Then it struck me: why would <em>that</em> API need to be the one people use? It can remain public, but maybe we can build
something <em>above</em> it to make it much easier to use. The initial idea of that API was to be an <a href="https://en.wikipedia.org/wiki/Domain-specific_language">EDSL</a>, but Rust has a lot
of limitations (for instance, <code>Eq</code> methods, <code>Ord</code> methods, etc. return <code>bool</code>, while I would need them to return
<code>Expr&lt;bool&gt;</code>). So… I came to the realization that the right way to do this would be to write a <em>real</em> EDSL: <a href="https://crates.io/crates/shades-edsl">shades-edsl</a>.</p>
<p>The idea of <a href="https://crates.io/crates/shades-edsl">shades-edsl</a> is super simple: remove everything that would make <a href="https://crates.io/crates/shades">shades</a> usable directly by a human, and
build a procedural macro that transforms regular Rust code into the API from <a href="https://crates.io/crates/shades">shades</a>. Basically, generate the <a href="https://crates.io/crates/shades">shades</a>
code inside the code of a procedural macro, which is what humans will use.</p>
<p>The way it works is pretty straight-forward to understand: you write your shading code in a <code>shades! { … }</code> block.
Everything in that block is reinterpreted and replaced by <a href="https://crates.io/crates/shades">shades</a> symbols: builders, method calls, etc. etc.</p>
<p>For instance, the following:</p>
<pre><code class="language-rust">let stage = shades! { VS |_input, _output, _env| {
  fn add(a: i32, b: i32) -&gt; i32 {
    a + b
  }

  fn main() {
    // do nothing
  }
}};
</code></pre>
<p>Will be reinterpreted as:</p>
<pre><code class="language-rust">let stage = shades::stage::StageBuilder::&lt;_, _, _&gt;::new_vertex_shader(…);
</code></pre>
<p>Where <code>…</code> is generated code (a lambda here). A couple of more example:</p>
<pre><code class="language-rust">  // in a shades! block
  if cond {
    body_if
  } else {
    body_else
  }
</code></pre>
<p>is transformed into:</p>
<pre><code class="language-rust">  __scope.when(cond, |__scope| {
    #body_if // body_if is also transformed
  }).or_else(|__scope| {
    #body_else
  });
</code></pre>
<h1>Feature list</h1>
<ul>
<li>Constant declarations: the regular Rust <code>const</code> declarations work and declare constant items in the shading code.</li>
<li>Function declarations: two kind of declarations are supported:
<ul>
<li>Any kind of functions declared with <code>fn</code> will create function declarations and return a function handle (so that you
can call the function).</li>
<li>A function called <code>main</code> will not return a function handle, but instead will inject the function declaration and
will return the <code>Stage</code>. It is the only way to produce a <code>Stage</code> so you have the certainty that a <code>Stage</code> contains
the <code>main</code> function.</li>
</ul>
</li>
</ul>
<h1>How it works</h1>
<p><a href="https://crates.io/crates/shades-edsl">shades-edsl</a> is implemented as a procedural macros, <code>shades!</code>, which defines a <em>shader stage</em>. It’s basically a
monadic computation (using <code>shades::stage::StageBuilder</code> behind the scene). Once you write your <code>fn main()</code> function,
the builder simply generates the <code>Stage</code> object for you (using <code>StageBuilder::main_fun</code>). In terms of architecture, it’s
a pretty simple one:</p>
<ol>
<li>I use the <a href="https://crates.io/crates/syn">syn</a> crate to parse the content of the <code>shades!</code> invocation (<code>TokenStream</code>).</li>
<li>I then <em>mutate</em> the generated AST to optimize and inject some logic in your code.</li>
<li>I marshal the AST back to Rust by generating the calls to the <a href="https://crates.io/crates/shades">shades</a> API, and write the Rust code back to the
<code>TokenStream</code></li>
</ol>
<h2>Parsing the Rust code</h2>
<p>Parsing the Rust code uses <a href="https://crates.io/crates/syn">syn</a>, a famous parsing crate used a lot by procedural macros. It was made to parse Rust-like
code, but it has some interesting use cases. I’m used to writing procedural macros in <a href="https://crates.io/crates/luminance-derive">luminance-derive</a> for instance
(<code>#[derive()]</code> annotations).</p>
<p>The idea is to take a Rust item and represent it as a data type. The important thing to know with <a href="https://crates.io/crates/syn">syn</a> is that it
supports nice error message reporting, so you are advised to store non-meaningful information, like braces, parens,
tokens, etc.</p>
<p>For instance:</p>
<pre><code class="language-rust">let x: Type = 3;
</code></pre>
<p>We can represent such an item (a <code>let</code> declaration) with the following <code>struct</code>:</p>
<pre><code class="language-rust">#[derive(Debug)]
pub struct VarDeclItem {
  let_token: syn::Token![let],
  name: syn::Ident,
  ty: Option&lt;(syn::Token![:], syn::Type)&gt;,
  assign_token: syn::Token![=],
  expr: syn::Expr,
  semi_token: syn::Token![;],
}
</code></pre>
<blockquote>
<p><code>syn::Token!</code> is a type macro that resolves to the right type name for the given symbol.</p>
</blockquote>
<p>Parsing in <a href="https://crates.io/crates/syn">syn</a> uses the <code>Parse</code> trait, and is more intimidating than hard to use. This is the whole parser for
<code>VarDeclItem</code>:</p>
<pre><code class="language-rust">impl Parse for VarDeclItem {
  fn parse(input: syn::parse::ParseStream) -&gt; Result&lt;Self, syn::Error&gt; {
    let let_token = input.parse()?;
    let name = input.parse()?;

    let ty = if input.peek(Token![:]) {
      let colon_token = input.parse()?;
      let ty = input.parse()?;
      Some((colon_token, ty))
    } else {
      None
    };
    let assign_token = input.parse()?;
    let expr = input.parse()?;
    let semi_token = input.parse()?;

    Ok(Self {
      let_token,
      name,
      ty,
      assign_token,
      expr,
      semi_token,
    })
  }
}
</code></pre>
<p><code>ParseStream</code> is the current state of the input stream. As you can see with the type of <code>parse()</code>, it will return the
<code>Self</code> type or an error, so we can call <code>input.parse()?</code> inside the implementation if type inference knows what the type
is, and that the type implements <code>Parse</code>. In our case, <code>syn::Token![let]</code> and <code>syn::Ident</code> do implement <code>Parse</code>, so we
can parse them in two simple lines:</p>
<pre><code class="language-rust">    let let_token = input.parse()?;
    let name = input.parse()?;
</code></pre>
<p>Then, we need to look ahead to check whether there is a semicolon (<code>:</code>). If there is, then we can parse <code>syn::Token![:]</code>
and <code>syn::Type</code>, as both already implement <code>Parse</code>. Otherwise, we simply assume <code>None</code> as type ascription:</p>
<pre><code class="language-rust">    let ty = if input.peek(Token![:]) {
      let colon_token = input.parse()?;
      let ty = input.parse()?;
      Some((colon_token, ty))
    } else {
      None
    };
</code></pre>
<p>The rest is as simple as everything implements <code>Parse</code> as well:</p>
<pre><code class="language-rust">    let assign_token = input.parse()?;
    let expr = input.parse()?;
    let semi_token = input.parse()?;
</code></pre>
<p>We have everything we need to return the parsed declaration:</p>
<pre><code class="language-rust">    Ok(Self {
      let_token,
      name,
      ty,
      assign_token,
      expr,
      semi_token,
    })
</code></pre>
<p>Every Rust items are parsed like that. There are a couple of harder things. For instance, input can be split by parsing
matched delimiters, resulting in two disjoint inputs, or some items will have to be delimited with punctuation (a
token).</p>
<p>An example of disjoint input parsing while parsing <code>if</code> statements:</p>
<pre><code class="language-rust">#[derive(Debug)]
pub struct IfItem {
  if_token: Token![if],
  paren_token: Paren,
  cond_expr: Expr,
  brace_token: Brace,
  body: ScopeInstrItems,
  else_item: Option&lt;ElseItem&gt;,
}

// …

impl Parse for IfItem {
  fn parse(input: syn::parse::ParseStream) -&gt; Result&lt;Self, syn::Error&gt; {
    let if_token = input.parse()?;

    let cond_input;
    let paren_token = parenthesized!(cond_input in input);
    let cond_expr = cond_input.parse()?;

    let body_input;
    let brace_token = braced!(body_input in input);
    let body = body_input.parse()?;

    let else_item = if input.peek(Token![else]) {
      Some(input.parse()?)
    } else {
      None
    };

    Ok(Self {
      if_token,
      paren_token,
      cond_expr,
      brace_token,
      body,
      else_item,
    })
  }
}
</code></pre>
<p>Here, we use the <code>parenthesized!</code> macro that parses a matching pair of <code>()</code>. Once it has finished, the <code>input</code> has
advanced after the closing delimiter, but the second input (<code>cond_input</code> in our case) is scoped right after the opening
delimiter and right before the closing one. That allows us to parse “inside” the parenthesis, which is a weird interface
at first, but once you get used to it, it’s actually pretty smart and pretty fast. Notice how we parse the conditional
expression using <code>cond_input</code> and not <code>input</code> here:</p>
<pre><code class="language-rust">    let cond_input;
    let paren_token = parenthesized!(cond_input in input);
    let cond_expr = cond_input.parse()?;
</code></pre>
<p>Same thing with <code>{}</code> and the <code>braced!</code> macro:</p>
<pre><code class="language-rust">    let body_input;
    let brace_token = braced!(body_input in input);
    let body = body_input.parse()?;
</code></pre>
<p>An even more interesting example: parsing function definitions:</p>
<pre><code class="language-rust">#[derive(Debug)]
pub struct FunDefItem {
  fn_token: Token![fn],
  name: Ident,
  paren_token: Paren,
  args: Punctuated&lt;FnArgItem, Token![,]&gt;,
  ret_ty: Option&lt;(Token![-&gt;], Type)&gt;,
  brace_token: Brace,
  body: ScopeInstrItems,
}
</code></pre>
<p>Here, <code>args</code> is a punctuated sequence of <code>FnArgItem</code>, delimited with <code>,</code>. The parser is as follows:</p>
<pre><code class="language-rust">impl Parse for FunDefItem {
  fn parse(input: syn::parse::ParseStream) -&gt; Result&lt;Self, syn::Error&gt; {
    let fn_token = input.parse()?;
    let name = input.parse()?;

    let args_input;
    let paren_token = parenthesized!(args_input in input);
    let args = Punctuated::parse_terminated(&amp;args_input)?;

    let ret_ty = if input.peek(Token![-&gt;]) {
      let arrow_token = input.parse()?;
      let ret_ty = input.parse()?;
      Some((arrow_token, ret_ty))
    } else {
      None
    };

    let body_input;
    let brace_token = braced!(body_input in input);
    let body = body_input.parse()?;

    Ok(Self {
      fn_token,
      name,
      paren_token,
      args,
      ret_ty,
      brace_token,
      body,
    })
  }
}
</code></pre>
<p>The argument part uses <code>parenthesized!</code> to parse the all arguments definitions, and then, with a single argument parser
(identifier plus type: <code>ident: Type</code>), we can simply ask to parse many by using comma punctuation. This is done with
<code>parse_terminated</code>:</p>
<pre><code class="language-rust">    let args_input;
    let paren_token = parenthesized!(args_input in input);
    let args = Punctuated::parse_terminated(&amp;args_input)?;
</code></pre>
<p>Parsing with <a href="https://crates.io/crates/syn">syn</a> is a bit weird at first (I’m used to <a href="https://crates.io/crates/nom">nom</a> because of <a href="https://www.khronos.org/opengl/wiki/OpenGL_Shading_Language">glsl</a>), but in the end, I think it’s a pretty
funny and interesting interface. There is no combinator such as alternative parser selection, but that’s probably for
the best, because the <em>peek</em> / <em>look ahead</em> interface forces you to write fast parsers. I really like it.</p>
<h2>Mutating the AST</h2>
<p>Mutating the AST is a top-down operation that consists in transforming types and expressions. The goal is to <em>lift</em>
items up the shades EDSL. A function such as:</p>
<pre><code class="language-rust">fn add(a: i32, b: i32) -&gt; i32
</code></pre>
<p>Needs to be <em>representable</em> in the <code>shades</code> AST. For this reason, <code>a</code>, <code>b</code> and the return type have to change types.
Changing the type implies going from <code>T</code> to <code>Expr&lt;T&gt;</code>. That operation is done in a <code>mutate(&amp;mut self)</code> method on each
syntax items parsed from the previous stage, and each of <code>mutate</code> invocation might call <code>mutate</code> on items they contain
(such as punctuated items, collections of instructions / statements, etc.).</p>
<p>The only interesting part to discuss here is probably how we can lift values in complex expressions such as
<code>a + (b * c)</code>. This is done with the concept of mutable visitors, defined in <a href="https://crates.io/crates/syn">syn</a> with the <code>visit-mut</code> feature gate.
A mutable visitor is simply an object that implements a trait (<code>VisitMut</code>), which defines one method for each type of
items in the AST that might exist. For instance, there is <code>visit_expr_mut(&amp;mut self, e: &amp;mut syn::Expr)</code> and a
<code>visit_type_mut(&amp;mut self, e: &amp;mut syn::Type)</code>. All <code>visit_*</code> methods have a default implementation doing nothing, so
that you are left with implementing only the ones that matter for you.</p>
<p>An example of what I do with types:</p>
<pre><code class="language-rust">struct ExprVisitor;

impl VisitMut for ExprVisitor {
  fn visit_type_mut(&amp;mut self, ty: &amp;mut Type) {
    *ty = parse_quote! { shades::expr::Expr&lt;#ty&gt; };
  }
}
</code></pre>
<blockquote>
<p><code>parse_quote!</code> is a <a href="https://crates.io/crates/syn">syn</a> macro that is akin to <code>quote!</code> from <a href="https://crates.io/crates/quote">quote</a>, but can infer the return type to parse to the
right type. Here, the returned type is inferred to be <code>syn::Type</code>.</p>
</blockquote>
<p>An interesting thing with <a href="https://crates.io/crates/shades-edsl">shades-edsl</a>: because users will provide their inputs, outputs and environment types in the
types of <code>Stage</code>, we do not want to wrap those in <code>Expr</code>, because they already contain <code>Expr</code> values, as seen earlier.
For this reason, the syntax of types of arguments was augmented with the possibility to add a dash (<code>#</code>) in behind the
types. If such a symbol is found, the type is assumed <em>unlifted</em> and shouldn’t be lift. For instance:</p>
<pre><code class="language-rust">let a: i32 = …; // will become Expr&lt;i32&gt;
let b: #Foo = …; // will remain Foo
</code></pre>
<h2>Generating the <code>shades</code> code</h2>
<p>Generating the <a href="https://crates.io/crates/shades">shades</a> AST is just a matter of traversing the parsed and mutated AST and using the <a href="https://crates.io/crates/quote">quote</a> crate and
its <code>quote!</code> macro to write the Rust code we want. The only missing part is how to bootstrap a <code>StageBuilder</code>, which is
done with some more parsing. Invocation looks like this:</p>
<pre><code class="language-rust">  let stage = shades! { vertex |input: TestInput, output: TestOutput, env: TestEnv| {
</code></pre>
<p>The first keyword is <code>vertex</code> here, but it could be <code>fragment</code>, <code>geometry</code> or any kind of shader stage (that needs to be
documented). Then, you have a lambda. I chose that syntax because a shader stage can be imagined as a function of its
input, output and environment. I hesitated with returning the output, but it would make things harder for some stages
(i.e. geometry shaders), where they can write to the same output several times. In that lambda, every type is
automatically marked as unlifted (so <code>input: TestInput</code> and <code>input: #TestInput</code> are the same).</p>
<p>Once transformed, this single line is akin to something like this:</p>
<pre><code class="language-rust">shades::stage::StageBuilder::new_vertex_shader(|mut __builder: shades::stage::StageBuilder::&lt;#user_in_ty, #user_out_ty, #user_env_ty&gt;, #input, #output| {
</code></pre>
<p>Everything else works the same way. Some fun example:</p>
<p>The generated code when the <code>main</code> function is found:</p>
<pre><code class="language-rust">    if name.to_string() == &quot;main&quot; {
      let q = quote! {
        // scope everything to prevent leaking them out
        {
          // function scope
          let mut __scope: shades::scope::Scope&lt;()&gt; = shades::scope::Scope::new(0);

          // roll down the statements
          #body

          // create the function definition
          use shades::erased::Erased as _;
          let erased = shades::fun::ErasedFun::new(Vec::new(), None, __scope.to_erased());
          let fundef = shades::fun::FunDef::&lt;(), ()&gt;::new(erased);
          __builder.main_fun(fundef)
        }
      };

      q.to_tokens(tokens);
      return;
    }
</code></pre>
<p>If items:</p>
<pre><code class="language-rust">impl ToTokens for IfItem {
  fn to_tokens(&amp;self, tokens: &amp;mut proc_macro2::TokenStream) {
    let cond = &amp;self.cond_expr;
    let body = &amp;self.body;

    let q = quote! {
    __scope.when(#cond, |__scope| {
      #body
    }) };

    q.to_tokens(tokens);

    match self.else_item {
      Some(ref else_item) =&gt; {
        let else_tail = &amp;else_item.else_tail;
        quote! { #else_tail }.to_tokens(tokens);
      }

      None =&gt; {
        quote! { ; }.to_tokens(tokens);
      }
    }
  }
</code></pre>
<p>Else tail items, which allows to add an <code>if</code> again (<code>else if</code>) or stop to <code>else</code> only. You will notice in <code>ToTokens</code>
implementation of <code>IfItem</code> that there is no <code>;</code> added at the end of the <code>.when()</code> call unless there is no <code>else</code>
following, and that rule is kept enforced in <code>ToTokens for ElseTailItem</code> as well. See, static analysis at compile-time!:</p>
<pre><code class="language-rust">impl ToTokens for ElseTailItem {
  fn to_tokens(&amp;self, tokens: &amp;mut proc_macro2::TokenStream) {
    match self {
      ElseTailItem::If(if_item) =&gt; {
        if_item.to_tokens_as_part_of_else(tokens);
      }

      ElseTailItem::Else { body, .. } =&gt; {
        let q = quote! {
          .or(|__scope| { #body });
        };

        q.to_tokens(tokens);
      }
    }
  }
}
</code></pre>
<h1>Writers</h1>
<p>Once you get a <code>Stage</code>, you can use a <em>writer</em> to output a <code>String</code>, so that <a href="https://crates.io/crates/shades">shades</a> is compatible with any kind of
library / framework / engine that expects a string-based shader source.</p>
<p>For example, for GLSL:</p>
<pre><code class="language-rust">  let output = shades::writer::glsl::write_shader_to_str(&amp;vertex_shader).unwrap();
</code></pre>
<h1>Interaction with <code>luminance</code></h1>
<p>Eh, I wrote both crates, so obviously <a href="https://crates.io/crates/luminance">luminance</a> has / will have a special support of <a href="https://crates.io/crates/shades">shades</a>. Remember the definition
of <code>Environment</code> above? It’s similar to <code>Inputs</code> and <code>Outputs</code>, too. <a href="https://crates.io/crates/luminance-derive">luminance-derive</a> already provides some derive
macros to automatically implement some traits, and it will provide a support to implement <code>Inputs</code>, <code>Outputs</code> and
<code>Environment</code> as well, without having to think about the indexes.</p>
<h1>Caveats and alternative work</h1>
<p>Of course, if something is not implemented correctly (a bug in <a href="https://crates.io/crates/shades-edsl">shades-edsl</a>, a bug in <a href="https://crates.io/crates/shades">shades</a>), debugging the output
shaders will be harder, because everything is generated (names included) on the target side. It’s something to keep in
mind.</p>
<p>You should also have a look at some similar yet different approaches:</p>
<ul>
<li><a href="https://github.com/EmbarkStudios/rust-gpu">rust-gpu</a> from EmbarkStudios. It’s a similar project but instead of
parsing Rust code, they wrote a compile chain (that you need to install via <code>rustup</code>) and compile your code with. It’s
an interesting point but I wanted something more typed with a more seamless integration to the rest of my ecosystem
(i.e. <a href="https://crates.io/crates/luminance">luminance</a>).</li>
<li><a href="https://www.w3.org/TR/WGSL">wgsl</a>, a new shading language for the WebGPU.</li>
<li><a href="https://crates.io/crates/glsl-quasiquote">glsl-quasiquote</a>: you can write GLSL code in it and get it parsed at compile-time. However, it will use the <a href="https://www.khronos.org/opengl/wiki/OpenGL_Shading_Language">glsl</a>
crate, which is currently a parser only, and it will not do any kind of semantic analysis. Another crate has to be
written for that, and it doesn’t exist to my knowledge (and I won’t write it because it’s a lot of work and I don’t
think it’s the right approach, unless you want to write a GLSL compiler, which I don’t).</li>
</ul>
<p>Keep the vibes!</p>
<blockquote>
<p><a href="https://www.reddit.com/r/rust_gamedev/comments/wbxpp4/a_shading_language_edsl/">Reddit discussion</a></p>
</blockquote>


guid: https://strongly-typed-thoughts.net/blog/splines-introductionSun, 05 Aug 2018 19:00:00 GMT
title: Easy interpolation across splines
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/splines-introduction
feedurl: https://phaazon.net/blog/feed
content:
<p>This article introduces the <a href="https://crates.io/crates/splines">splines</a> crate.</p>
<h1>Preliminary work</h1>
<p>I’ve been using interpolation (linear, bilinear, trilinear, B-splines, etc.) for a while now because
of my demoscene productions. My <a href="https://crates.io/crates/spectra">spectra</a> crate features some
interpolation primitives and I’ve been wanting to move them out of spectra for a while.</p>
<p>Basically, my need is simple:</p>
<ul>
<li>I use interpolation for <em>a lot</em> of situations: animation, procedural generation, editing,
transitions, etc.</li>
<li>I need several interpolation implicit methods / spaces:
<ul>
<li>Constant/step.</li>
<li>Linear / bilinear / trilinear.</li>
<li>Cosine.</li>
<li>Cubic Hermite spline.</li>
</ul>
</li>
<li>I also need splines with control points (knots):
<ul>
<li>Catmull-Rom.</li>
<li>Bézier.</li>
</ul>
</li>
</ul>
<p>That’s a lot of things. For what it’s worth, I’ll just use as reference a library I wrote years ago
in Haskell, <a href="https://hackage.haskell.org/package/smoothie">smoothie</a>, doing exactly that.</p>
<p>However, because I didn’t want to bloat the ecosystem, I had a look around to see whether I could
simply drop my code and add a simple dependency instead of extracting things from spectra and
creating, again, new crates. Here are the reasons why.</p>
<h2><code>bspline</code></h2>
<p>This crate is about <a href="https://en.wikipedia.org/wiki/B-spline">B-spline</a> only and the interface is highly unsafe (panics if out of control
points for instance). My current spectra code is already safer than that, so no reason to lose
features here.</p>
<h2><code>spline</code></h2>
<p>This is… a… placeholder crate – i.e. it doesn’t have anything exposed. People do this to book a
crate name for later. I highly dislike that and people doing this should really be warned not to do
this, because they prevent others – who actually have some code to put there – from using the name.
Plus, for this <code>spline</code> crate, I had already have a look years ago. In years, the so-called “author”
hasn’t even uploaded anything and the name is just reserved… for nothing.</p>
<p>I highly suggest people from the Rust team to forbid people from doing this. This is just useless,
childish and brings zero value to the community. Just remove that crate as no one has ever had <em>the
chance</em> to ever depend on it.</p>
<h2><code>trajectory</code></h2>
<p>This is an interesting crate, but it’s way too much specialized to me. It could be very interesting
to use it for camera and objects paths, but I also need splines for a lot of other situations, so
this is too much restrictive (you can see functions like <code>acceleration</code>, <code>position</code>, <code>velocity</code> that
works only for <code>T: Float</code>).</p>
<h2><code>nbez</code></h2>
<p>A – looks like – very comprehensive Bezier crate, but doesn’t provide anything for all the other
interpolation I need.</p>
<h1>Let’s build yet another crate!</h1>
<p>Looks like the crate I’m looking for doesn’t exist yet – oh yes it does: it’s spectra!, but I want
a crate that does interpolation only.</p>
<p>I could be using some of the crates listed above, add them as a bunch of dependencies to a <em>glue
crate</em> and just expose the whole thing. However, I’m not sure they will compose quite correctly and
I might just end up re-coding the whole thing.</p>
<p>So the crate I’ll be building will be about interpolation. You will find step (constant)
interpolation, linear / bilinear / trilinear interpolation, cosine, cubice, Catmull-Rom and Bézier
spline interpolations. The crate will be as abstract as possible to enable developers to plug in
their own types. Because the <code>spline</code> name is already taken – duh! – I’ll use the <code>splines</code> name…</p>
<blockquote>
<p>I’ll take my chance to try to take over <code>spline</code> first, though. I really don’t like that kind of
things in an ecosystem, it’s just unaesthetic.</p>
</blockquote>
<blockquote>
<p>Note²: Ok, I tried, and seems like it’s more complicated than I thought. I’ll just go with the
<code>splines</code> name then.</p>
</blockquote>
<p>So this blogpost serves as an introduction to <a href="https://crates.io/crates/splines">splines</a>. I added a few unit tests to start off with
and a very few examples in the documentation. Most of the interface is <em>sufficient</em> for now but more
complex needs might show up. If you have any, please shoot an issue or even better, open a PR!</p>
<p>Keep the vibes and happy hacking!</p>


guid: https://strongly-typed-thoughts.net/blog/rust-no-dropTue, 22 May 2018 22:22:00 GMT
title: Moving out of a Drop struct in Rust?
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/rust-no-drop
feedurl: https://phaazon.net/blog/feed
content:
<p>Today, I’ve come across a very interesting problem in Rust. This problem comes from a very specific
part of <a href="https://crates.io/crates/luminance">luminance</a>.</p>
<p>Rust has this feature called <em>move semantics</em>. By default, the <em>move semantics</em> are enforced. For
instance:</p>
<pre><code>fn foo_move&lt;T&gt;(x: T)
</code></pre>
<p>This function says that it expects a <code>T</code> and need to move it in. If you don’t want to move it, you
have to <em>borrow</em> it. Either immutable or mutably.</p>
<pre><code>fn foo_borrow&lt;T&gt;(x: &amp;T)
fn foo_borrow_mut&lt;T&gt;(x: &amp;mut T)
</code></pre>
<p>Now, when you move a value around, it’s possible that you don’t use that value anymore. If you look
at the <code>foo_move</code> function, is very likely that we’ll not use that <code>T</code> object after the function has
returned. So Rust knows that <code>x</code> will go out of scope and then its memory must be invalidated and
made available to future values. This is very simple as it will happen on the stack, but it can get
a bit tricky.</p>
<p>Rust doesn’t have the direct concept of <em>destructors</em>. Or does it? Actually, it does. And it’s
called the <code>Drop</code> trait. If a type implements <code>Drop</code>, any value of this type will call a specific
function for disposal when it goes out of scope. This is akin to running a destructor.</p>
<pre><code>struct Test;

impl Drop for Test {
  fn drop(&amp;mut self) {
    println!(&quot;drop called!&quot;);
  }
}
</code></pre>
<p>Whenever a value of type <code>Test</code> goes out of scope / must die, it will call its <code>Drop::drop</code>
implementation and then its memory will be invalidated.</p>
<p>Ok, now let’s get into my problem.</p>
<h1>Moving out of a field</h1>
<p>Consider the following code:</p>
<pre><code>struct A;

struct B;

struct Foo {
  a: A,
  b: B
}

impl Foo {
  fn take(self) -&gt; (A, B) {
    (self.a, self.b)
  }
}

fn main() {
  let foo = Foo { a: A, b: B };
  let (a, b) = foo.take();
}
</code></pre>
<p>The implementation of <code>Foo::take</code> shows that we <em>move out of <code>Foo</code></em> – two, actually. We move the
<code>a: A</code> field out of <code>Foo</code> and <code>b: B</code> out of <code>Foo</code>. Which means that we <em>destructured</em> <code>Foo</code>. The
call to <code>Foo::take</code> in the <code>main</code> function demonstrates how you would use that function.</p>
<p>Now consider this. <code>A</code> and <code>B</code> here are opaque type we don’t want to know the implementation. But
keep in mind they represent <em>scarce resources</em> – like UUID to important resources we must track in
the <code>Foo</code> object. Consider this new code:</p>
<pre><code>struct A;

struct B;

struct Foo {
  a: A,
  b: B
}

impl Drop for Foo {
  fn drop(&amp;mut self) {
    // something tricky with self.a and self.b
  }
}

impl Foo {
  fn take(self) -&gt; (A, B) {
    (self.a, self.b) // wait, that doesn’t compile anymore!
  }
}
</code></pre>
<p>You would get the following error:</p>
<blockquote>
<p>error[E0509]: cannot move out of type <code>Foo</code>, which implements the <code>Drop</code> trait</p>
</blockquote>
<p>As you can see, Rust doesn’t allow you to move out of a value which type implements <code>Drop</code>, and this
is quite logical. When <code>Foo::take</code> returns, because of <code>self</code> going out of scope, it must call
its <code>Drop::drop</code> implementation. If you have moved out of it – both <code>a: A</code> and <code>b: B</code> fields, the
<code>Drop::drop</code> implementation is now a complete <a href="https://en.wikipedia.org/wiki/Undefined_behavior">UB</a>. So Rust is right here and doesn’t allow you to
do this.</p>
<p>But imagine that we <strong>have</strong> to do this. For insance, we need to hand over both the scarce resources
<code>a</code> and <code>b</code> to another struct (in our case, a <code>(A, B)</code>, but you could easily imagine a better type
for this).</p>
<p>There’s a way to, still, implement <code>Foo::take</code> with <code>Foo</code> implementing <code>Drop</code>. Here’s how:</p>
<pre><code>impl Foo {
  fn take(mut self) -&gt; (A, B) {
    let a = mem::replace(&amp;mut self.a, unsafe { mem::uninitialized() });
    let b = mem::replace(&amp;mut self.b, unsafe { mem::uninitialized() });
    
    mem::forget(self);
    
    (a, b)
  }
}
</code></pre>
<p>There’s a few new things going on here. First, the <code>mem::replace</code> function takes as first parameter
a mutable reference to something we want to replace by its second argument and returns the initial
value the mutable reference pointed to. Because <code>mem::replace</code> doesn’t deinitialize any of its
argument, I like to think of that function as a <em>move-swap</em>. It just moves out the first argument
and put something in the hole it has made with its second argument.</p>
<p>The <code>unsafe</code> function, <code>mem::uninitialized</code>, is a bit special as it bypasses Rust’s normal
memory-initialization and actually doesn’t do anything. However, there’s no specification about what
this function really does and you shouldn’t be concerned. What you just need to know is that this
function just gives you <em>uninitialized</em> memory.</p>
<p>However, if we just returned <code>(a, b)</code> right after the two <code>mem::replace</code> calls, the <code>Drop::drop</code>
implementation would run at the end of the function, and it would run <em>badly</em>, because of the now
uninitialized data you find in the <code>self.a</code> and <code>self.b</code> fields. It could still be the old values.
Or it could be garbage. We don’t care. We mustn’t read from there. Rust provide a solution to this,
and this is the <code>mem::forget</code> function. This function is like dropping your value… without calling
its <code>Drop::drop</code> implementation. It’s an escape hatch. It’s a way to invalidate a value and in our
case that function is super handy because <code>self</code> now contains uninitialized data!</p>
<p>And here we have a complete working <code>Foo::take</code> function.</p>
<h1>So what’s the problem again?</h1>
<p>The problem is that I used to function I wish I didn’t:</p>
<ul>
<li><code>mem::uninitialized</code>, which is,
<a href="https://doc.rust-lang.org/std/mem/fn.uninitialized.html">as the docs state</a>,
<strong>incredibly hazardous</strong>.</li>
<li><code>mem::forget</code>, which is not unsafe in the Rust idea of safety, but still has some kind of very
nasty unsafety deep in it – it’s very easy to leak resources if you don’t pay extra attention.</li>
</ul>
<p>The problem is that Rust doesn’t allow you to move out of a value which implement <code>Drop</code> even if you
know that you’ll <code>forget</code> it. Rust doesn’t currently have a primitive / function to express both
principles at the same time: forgetting and moving out of a <code>Drop</code> value.</p>
<p>This had me to come to the realization that I would like an interface like this:</p>
<pre><code>fn nodrop&lt;T, F, A&gt;(
  x: T,
  f: F
) -&gt; A
where T: Drop,
      F: FnOnce(T) -&gt; A
</code></pre>
<p>This function would give you the guarantee that the closure will not <code>drop</code> its argument but would
still leak it, temporarily giving you the power to move out of it. You will also notice that if you
want to have <code>A = T</code>, it’s possible: that would mean you destructure a value and re-build it back
again inside the same type, which is still sound.</p>
<p>However, this function would make it very easy to leak any field you don’t move out. This is
actually the same situation as with <code>mem::forget</code>: if you forget to <code>mem::replace</code> a field before
calling <code>mem::forget</code>, you end up in the exact same situation.</p>
<p>I agree it’s a very niche need, and if you’re curious about why I need this, it’s because of how
<em>buffers</em> are handled in <a href="https://crates.io/crates/luminance">luminance</a>. The <code>Buffer&lt;T&gt;</code> type has an internal <code>RawBuffer</code> as field that
holds GPU handles and internal details and a <code>PhantomData&lt;T&gt;</code>. It’s possible to convert from
one type to the other but both implement <code>Drop</code>, so you have to implement the hack I just presented
in this blog post to make both conversions. With my <code>nodrop</code> function, the code would be way
easier, without needing to unfold <code>unsafe</code> blocks. Something like this would be possible:</p>
<pre><code>pub fn to_raw(self) -&gt; RawBuffer {
  mem::nodrop(self, |buffer| buffer.raw) // combine mem::forget and mem::replace + uninitialized
}
</code></pre>
<p>What do you peeps think of such a function? For sure that would add possible easy leaks, that’s a
drawback I find very difficult not to agree with. However, I just wanted to share my little
experience of Tuesday evening.</p>
<p>That’s all for me. Thanks for having read me, I love you. Keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/cargo-sync-readmeMon, 25 Feb 2019 18:50:00 GMT
title: New cargo subcommand: sync-readme
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/cargo-sync-readme
feedurl: https://phaazon.net/blog/feed
content:
<p>Have you ever struggled with welcoming people on your project pages, be them on <a href="https://github.com">GitHub</a> or
<a href="https://docs.rs">docs.rs</a>? One major problem we’re facing when developping crates is that we need to maintain
several places where documentation is important:</p>
<ul>
<li>The <code>lib.rs</code> / <code>main.rs</code> files, that get rendered as front page on <a href="https://docs.rs">docs.rs</a>. This is important
because we want people to have nice onboardings when hitting the official documentations of our
crates.</li>
<li>The README file — often <code>README.md</code>, that gets automatically rendered as HTML when you end up on
the project’s <a href="https://github.com">GitHub</a> page. This page should provide as many as possible hints and information
about what the project is and how to use it.</li>
</ul>
<p>However, sometimes, I just don’t want to bother writing twice the same thing and honestly, I don’t
really know whether the README file is a good place to write an onboarding section, since that
should also be included in your Rust documentation on <a href="https://docs.rs">docs.rs</a>.</p>
<p>So… I’ve been thinking of a way to fix that, without really investing too much time in it. But
lately, I came to the realization that I often have this pattern:</p>
<ol>
<li>Write the onboarding documentation in my <code>lib.rs</code> or <code>main.rs</code>. For instance, I’m pretty proud
of the <a href="https://docs.rs/warmy">onboarding documentation of warmy</a>. It’s exhaustive, easy for newcomers, it has plenty
of links and explanations and it renders pretty nice on <a href="https://docs.rs">docs.rs</a>.</li>
<li>Write a header in the README file with badges etc. and then copy the Rust onboarding
documentation in the readme.</li>
</ol>
<p>Here, (2.) is a manual and annoying task: I open my editor, make a block selection of the
documentation, store it in a buffer, apply a smart macro on it to remove the Rust annotation and
then paste the result in the README after having purged it from the previous documentation… That’s
tedious and not very interesting.</p>
<h1><code>cargo sync-readme</code> to the rescue!</h1>
<p>So, yesterday, I decided to start working on a small tool that would automate all this for me. The
idea is the following:</p>
<ul>
<li><code>cargo sync-readme</code> synchronizes your README (the file specified by the <code>readme</code> key in your
<code>Cargo.toml</code>, or just <code>README.md</code> by default) with the entrypoint of your library or binary
crate (by default, <code>lib.rs</code> or <code>main.rs</code>, or what is defined at the <code>path</code> key in your
manifest).</li>
<li>In order to perform the synchronization, you need to have put a special marker so that the
command is instructed where to put the synchronized documentation.
<ul>
<li>The <code>&lt;!-- cargo-sync-readme --&gt;</code> marker must lie on a single line where you want the
documentation to be inserted in your README.</li>
<li>Post generation, this marker will disappear and will get replaced by the synchronized
documentation, surrounded by two markers: <code>&lt;!-- cargo-sync-readme start&gt;</code> and
<code>&lt;!-- cargo-sync-readme end --&gt;</code>.</li>
</ul>
</li>
</ul>
<p>This is really all you have to do. <code>cargo sync-readme</code> will take care of the rest for you. There’re
two hypothesis that the command requires to be true, though:</p>
<ul>
<li>Your README file should be a Markdown-formatted file. It doesn’t have to be, but since Rust’s
documentation is written in Markdown, it should be.</li>
<li>You use the <code>//!</code> annotation to write your documentation. This is currently how
<code>cargo sync-readme</code> works. It doesn’t have to be solely using this annotation on longer term,
but currently, this is the only annotation supported. More can be added if needed.</li>
</ul>
<p>Basically, insert the marker once, and run <code>cargo sync-readme</code>. Every time you change your Rust
documentation, just call <code>cargo sync-readme</code> once again to synchronize the documentation in your
README file.</p>
<h2>On workspace crates</h2>
<p>Currently, <code>cargo sync-readme</code> doesn’t work with workspace crates. You will have to go into each of
the workspace’s members to run <code>cargo sync-readme</code> if you want to synchronize their respective
READMEs.</p>
<h1>Conclusion</h1>
<p><code>cargo sync-readme</code> is already available on <a href="https://crates.io/crates/cargo-sync-readme">crates.io</a> for you to use. You can install it as a
development tool with the following command:</p>
<pre><code>cargo install cargo-sync-readme
</code></pre>
<blockquote>
<p>Disclaimer: after having published <code>cargo-sync-readme</code>, I was told that there is already another
cargo plugin, <a href="https://crates.io/crates/cargo-readme">cargo-readme</a>, that already does that. Indeed, that crate does more or less the
same job. However, the way it does it is very different. First, it uses a template file while
<code>cargo-sync-readme</code> lets you use your README file without having to depend on a template. Also,
<a href="https://crates.io/crates/cargo-readme">cargo-readme</a> has special semantics in its template (like {{crate_name}}, etc.) while
<code>cargo-sync-readme</code> is simpler and just requires a single marker. To sum up: <code>cargo-readme</code> is
heavier and is likely to require you to break your file into two separate files but contains more
customization options while <code>cargo-sync-readme</code> only requires a single line in your README and
will synchronize from within that same file.</p>
</blockquote>
<p>Feel free to comment, open issues, drink beers, share that <a href="https://phaazon.net/media/uploads/bear_sausage.mp4">awesome bear driving a sausage podracer</a>
and most of all… keep the vibes!</p>
<ul>
<li><a href="https://github.com/phaazon/cargo-sync-readme"><code>cargo-sync-readme</code> on GitHub</a></li>
<li><a href="https://crates.io/crates/cargo-sync-readme"><code>cargo-sync-readme</code> on crates.io</a></li>
</ul>


guid: https://strongly-typed-thoughts.net/blog/glsl-quasiquotingFri, 05 Oct 2018 19:00:00 GMT
title: GLSL quasiquoting in Rust!
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/glsl-quasiquoting
feedurl: https://phaazon.net/blog/feed
content:
<p>A year ago, <a href="https://phaazon.net/blog/glsl_crate">I announced the glsl crate</a>, a crate used to parse
GLSL450-formated sources (and possibly GLSL460 since it’s almost the same thing, but a feature flag
is needed for that). I didn’t get much feedback about it but I’ve been using it in my <a href="https://crates.io/crates/cheddar">cheddar</a>
crate with a great of success (along with <a href="https://crates.io/crates/spectra">spectra</a>).</p>
<p>The <a href="https://crates.io/crates/glsl">glsl</a> crate enables you to parse a GLSL-formatted input (raw string or byte slice) into an AST
you can manipulate in Rust.</p>
<p>It’s very likely you will never have to work with that AST representation directly. Instead, you
might be interested in all the possible outcomes you can get from using the <a href="https://crates.io/crates/glsl">glsl</a> crate:</p>
<ul>
<li>Parse a file, like a <code>scene.glsl</code>, and get its AST representation in Rust.</li>
<li>Pass that AST to a <em>writer</em>, for instance, a <a href="https://en.wikipedia.org/wiki/Source-to-source_compiler">transpiler</a>, to convert from the GLSL to <a href="https://www.khronos.org/spir">SPIR-V</a>
or <a href="https://docs.microsoft.com/en-us/windows/desktop/direct3dhlsl/dx-graphics-hlsl">HLSL</a>.</li>
<li>Algorithmically enhance or enrich the GLSL AST without having to cope with raw strings.</li>
<li>Extract information from the GLSL source code (reflection).</li>
<li>Etc.</li>
</ul>
<p>Lately, in <a href="https://crates.io/crates/spectra">spectra</a>, I’ve been working on a more abstract way to write shaders in GLSL. The idea is
that I need to ship the library with some GLSL. The typical way people do this is by embedding a
static string into your library / application, like so:</p>
<pre><code>const SOME_VERTEX_SHADER: &amp;str =
&quot;void main() {
  // …
}&quot;;
</code></pre>
<blockquote>
<p>You can also use the <code>include_str!</code> standard macro to read it from a file at compile-time.</p>
</blockquote>
<p>The <a href="https://crates.io/crates/cheddar">cheddar</a> crate provides you with an enriched version of GLSL (the Cheddar language), and the
integration in <a href="https://crates.io/crates/spectra">spectra</a> is pretty much seamless. The GLSL is parsed from Cheddar, transpiled to the
GLSL AST and then the whole thing is passed <a href="https://crates.io/crates/luminance">luminance</a> that acts as a rendering backend of
<a href="https://crates.io/crates/spectra">spectra</a>. Becauses some AST transformation is needed, the <a href="https://crates.io/crates/glsl">glsl</a> crate is used in <a href="https://crates.io/crates/cheddar">cheddar</a>, and
some GLSL must be added in order to provide a smoother and nicer experience within <a href="https://crates.io/crates/spectra">spectra</a>.</p>
<p>The problem kicks in that transformation part. Since Cheddar is parsed directly into memory as an
AST (via the <a href="https://crates.io/crates/cheddar">cheddar</a> crate), how could we transform and add things to values from <a href="https://crates.io/crates/cheddar">cheddar</a>?</p>
<h1>The hard and manual way: the constructor syntax</h1>
<p>Pretty much all of the <a href="https://crates.io/crates/glsl">glsl</a> crate is public. There’s no invariant, so no need to hide anything.
It’s possible to create, for instance, a <code>AssignmentExpr</code> by simply calling its constructor syntax:</p>
<pre><code>let assignment_expr = AssignmentExpr {
  // fields here
  };
</code></pre>
<p>This is the regular way to go, but if you look at some types, such as the
<a href="https://docs.rs/glsl/0.9.2/glsl/syntax/enum.SimpleStatement.html"><code>SimpleStatement</code></a>, you’ll get
that it can get very noisy and verbose very quickly. So that’ll work, but that’ll also be very
incomfortable to work with.</p>
<p>The advantage of constructing and transforming that way is that you can destructure and
pattern-match ASTs.</p>
<h1>Enter the nice solution: quasiquoting</h1>
<p>As a Haskeller, I’ve been very frustrated by that construction problem, thinking that there must be
an easier and less noisy way to do so. That reminds me of all the EDSLs I wrote in my Haskeller life
and some very, <em>very</em> sweet embedded languages, like SQL, C or Java, directly into Haskell. That is
done via something we call <a href="https://wiki.haskell.org/Quasiquotation">quasiquoting</a>. Quasiquoting is a general concept but in our case,
applies to embedded languages.</p>
<p>The concept is simple: you write in a foreign language in a <em>quasiquoter</em> in your host language
(Haskell or Rust). The compiler then recognizes the quasiquoter and <strong>executes some code at compile
time</strong>. Once it’s done, it replaces the quasiquoted statement with the result of its computation.</p>
<p>One good example of quasiquoting is <a href="http://hackage.haskell.org/package/postgresql-query">postgresql-query</a>, a Haskell quasiquoting library for SQL. SQL
queries can be built with regular Haskell code but they can also be built out of a <em>quasiquoter</em>,
like so:</p>
<pre><code>-- some code elided for simplicity here
pqQuery [sqlExp|SELECT u.id, u.name, u.age
                FROM users AS u ^{cond}
                ORDER BY ^{ord}|]
</code></pre>
<p>The <code>[sqlExp|…|]</code> notation uses the quasiquoter syntax: the <code>sqlExp</code> is the quasiquoter to use, and
everything between the two pipes are the content / input to pass the quasiquoter. The compiler just
reads that, executes some code defined in the <a href="http://hackage.haskell.org/package/postgresql-query">postgresql-query</a> library and if it runs correctly,
the resulting expression is inserted directly at the place of use as if nothing has happened.
Brillant, right? :)</p>
<p>Having written <a href="https://crates.io/crates/glsl">glsl</a>, that seemed like a perfect – maybe even wanted and mandatory? – opportunity
to me to introduce GLSL quasiquoting into the Rust environment. Maybe it’ll also bring some people
onto the <a href="https://crates.io/crates/glsl">glsl</a> project – I really need hands, especially because the <a href="https://www.khronos.org/spir">SPIR-V</a> transpiler doesn’t
exist yet!</p>
<p>So, here you go. <a href="https://crates.io/crates/glsl-quasiquote/0.1.0">glsl-quasiquote-0.1</a> was released today! The crate provides you with two macros:
<code>glsl!</code> and <code>glsl_str!</code>. Both are procedural macros that requires a nightly compiler and the
<code>proc_macro_non_items</code> feature. They will both output a <a href="https://docs.rs/glsl/0.9.2/glsl/syntax/type.TranslationUnit.html"><code>TranslationUnit</code></a>, that represents a whole
shader AST. </p>
<p>The <code>glsl!</code> AST works the same way a quasiquoter from Haskell: instead of delimiting your GLSL code
with pipes, you write them inside parens or brackets, like so:</p>
<pre><code>glsl!{
  void main() {
  }
}
</code></pre>
<p>The <code>glsl_str!</code> macro does the exact same thing but expects a literal string as input. This is due
to the fact that quasiquoting in Rust is a bit tricky – who’s looking at me like I was about to
write an RFC? – because regular macros and procedural macros eat Rust tokens as inputs, forcing
their content to be Rust-friendly syntax. Well, GLSL is not: the <code>#version</code> and <code>#extension</code> pragmas
need newlines at the end, so the following will not work:</p>
<pre><code>glsl!{
  #version 330 core
  void main() {
  }
}
</code></pre>
<p>Instead, you must use the <code>glsl_str!</code> quasiquoter:</p>
<pre><code>glsl_str!{&quot;
  #version 330 core

  void main() {
  }
&quot;}
</code></pre>
<p>It’s a bit of an unaesthetica and pretty weird syntax but we don’t really have a choice (so far).</p>
<p>Whenever a GLSL is incorrectly formatted, you’ll get a compiler error. For instance, the following:</p>
<pre><code>glsl!{
  void main() 3 {
  }
}
</code></pre>
<p>generates the following <code>rustc</code> output:</p>
<pre><code>error: proc macro panicked
  --&gt; tests/lib.rs:26:11
   |
26 |     let _ = glsl!{
   |  ___________^
27 | |     void main() 3 {
28 | |     }
29 | |   };
   | |___^
   |
   = help: message: GLSL error: Err(ParseError { kind: Many1, info: &quot;void main (  ) 3 {  }&quot; })

error: aborting due to previous error
</code></pre>
<p>The crate was released today, so it needs a lot of love, attention and testing. I’m adding support
for it in <a href="https://crates.io/crates/spectra">spectra</a> for my experiences, so I should be updating it if I find anything wrong with it.</p>
<p>Feel free to use it, and have fun! I’m awaiting your feedback!</p>
<p>Keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/warmy-universal-jsonFri, 25 Jan 2019 00:50:00 GMT
title: Universal JSON support in warmy-0.11.1
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/warmy-universal-json
feedurl: https://phaazon.net/blog/feed
content:
<h1>Norbert the hot-dog</h1>
<p>I just added a new feature to <a href="https://crates.io/crates/warmy/0.11.1">warmy</a> to make it more accessible and easy to use. It concerns <a href="https://fr.wikipedia.org/wiki/JavaScript_Object_Notation">JSON</a>
and <a href="https://crates.io/crates/serde">serde</a>.</p>
<h2>Universal JSON support in warmy 0.11.1</h2>
<p>If you’ve been using <a href="https://crates.io/crates/warmy/0.11.1">warmy</a> a bit, you might be used to its traits and structures, like <code>Load</code>,
<code>Loaded</code>, <code>Load::reload</code>, <code>Res</code>, etc. All those concepts are mandatory to implement loading,
reloading and scarce resource sharing. Since version
<a href="https://github.com/phaazon/warmy/blob/master/CHANGELOG.md#07">0.7</a>, <a href="https://crates.io/crates/warmy/0.11.1">warmy</a> has got <em>loading and
reloading methods</em>. That feature enables you to have several <code>impl</code> for the same type of resource by
changing the method used to load. The default is <code>()</code> but you’re free to use any you want. The idea
is that you can call the <code>Store::get_by</code> or <code>Store::get_proxied_by</code> methods to specify which method
to use explicitly when loading and reloading a given resource.</p>
<p><a href="https://crates.io/crates/warmy/0.11.1">warmy</a> 0.11.1 uses that concept to provide a universal implementor for anything that implements the
<a href="https://docs.rs/serde/1.0.85/serde/trait.Deserialize.html"><code>Deserialize</code></a> trait from <a href="https://crates.io/crates/serde">serde</a>. Basically, once your type implements <a href="https://docs.rs/serde/1.0.85/serde/trait.Deserialize.html"><code>Deserialize</code></a>, you can
load and hot-reload values of this type by using the <a href="https://docs.rs/warmy/0.11.1/warmy/json/struct.Json.html"><code>Json</code></a> type from <a href="https://crates.io/crates/warmy/0.11.1">warmy</a>.</p>
<blockquote>
<p>Universal JSON implementors are available only if the <code>&quot;json&quot;</code> feature is enabled in <a href="https://crates.io/crates/warmy/0.11.1">warmy</a>. It’s
the case by default. Feel free to use <code>default-features = false</code> if you don’t want it.</p>
</blockquote>
<p><img src="https://phaazon.net/media/uploads/very_nice.gif" alt="" /></p>
<p>Here’s a short example of what it looks like to load and hot-reload a <code>Dog</code> using the universal JSON
feature:</p>
<pre><code>use serde::Deserialize;
use warmy::{Res, SimpleKey, Store, StoreOpt};
use warmy::json::Json;
use std::thread::sleep;
use std::time::Duration;

#[derive(Debug, Deserialize)]
#[serde(rename_all = &quot;snake_case&quot;)]
struct Dog {
  name: String,
  gender: Gender
}

#[derive(Clone, Copy, Debug, Deserialize, Eq, PartialEq)]
#[serde(rename_all = &quot;snake_case&quot;)]
enum Gender {
  Female,
  Male
}

fn main() {
  let mut store: Store&lt;(), SimpleKey&gt; = Store::new(StoreOpt::default()).unwrap();
  let ctx = &amp;mut ();

  let resource: Result&lt;Res&lt;Dog&gt;, _&gt; = store.get_by(&amp;SimpleKey::from_path(&quot;/dog.json&quot;), ctx, Json);

  match resource {
    Ok(dog) =&gt; {
      loop {
        store.sync(ctx);

        println!(&quot;Dog is {} and is a {:?}&quot;, dog.borrow().name, dog.borrow().gender);
        sleep(Duration::from_millis(1000));
      }
    }

    Err(e) =&gt; eprintln!(&quot;{}&quot;, e)
  }
}
</code></pre>
<p>This feature should help people adopt the crate and use it without worrying too much about the
actual implementation of <code>Load</code>.</p>
<p>If you’re interested in adding other methods, like YAML, XML or whatever, feel free to open a PR on
GitHub! I’ll be very happy to accept it. More documentation about the feature can be found in the
<a href="https://github.com/phaazon/warmy/tree/193af4bc6ee75699fea1ab2cf4ecff362b919899#universal-json-support">README</a> or the <a href="https://docs.rs/warmy/0.11.1/warmy/index.html#universal-json-support">online documentation</a>.</p>
<p>Keep the vibes.</p>


guid: https://strongly-typed-thoughts.net/blog/glsl-pest-part-1Sat, 17 Nov 2018 05:00:00 GMT
title: Introducing pest into glsl and hindsight about nom vs. pest (part 1)
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/glsl-pest-part-1
feedurl: https://phaazon.net/blog/feed
content:
<p>This is the first article out of a (I guess?!) series of upcoming articles about… <em>parsing</em>. More
specifically, I’ve been writing the <a href="https://crates.io/crates/glsl">glsl</a> crate for a while now and
the current, in-use parser is backed with <a href="https://crates.io/crates/nom">nom</a>. <a href="https://crates.io/crates/nom">nom</a> is a <a href="https://en.wikipedia.org/wiki/Parser_combinator">parser combinator</a> crate written
originally by <a href="https://github.com/geal">@geal</a> and there has been a lot of fuzz around <a href="https://crates.io/crates/nom">nom</a> vs. <a href="https://crates.io/crates/pest">pest</a> lately.</p>
<p>Soooooooooooo. Because <a href="https://crates.io/crates/glsl">glsl</a> is written with <a href="https://crates.io/crates/nom">nom</a> in its third iteration and because <a href="https://crates.io/crates/nom">nom</a> is now
at version <code>4</code>, I decided it was time to update the parser code of <a href="https://crates.io/crates/glsl">glsl</a>. I heard about the
comparison between <a href="https://crates.io/crates/pest">pest</a> and <a href="https://crates.io/crates/nom">nom</a> and decided to write an implementation with <a href="https://crates.io/crates/pest">pest</a>.</p>
<p>This is the first article of a series about how writing a <a href="https://crates.io/crates/pest">pest</a> looks like is fun compared to
writing the <a href="https://crates.io/crates/nom">nom</a> parser. I’ll post several articles as I advance and see interesting matter to
discuss and share.</p>
<ul>
<li>Second article <a href="https://phaazon.net/blog/glsl-pest-part-2">here</a></li>
</ul>
<h1>Oh my; I love PEG!</h1>
<p>First thing first: <a href="https://crates.io/crates/pest">pest</a> consumes a file at compile-time that contains a <a href="https://en.wikipedia.org/wiki/Parsing_expression_grammar">PEG</a> grammar defining the
format of input you want to parse. <a href="https://en.wikipedia.org/wiki/Parsing_expression_grammar">PEG</a> – which stands for <em>Parsing Expression Grammar</em> – is a
formal language that enables you to describe your own language with <em>rules</em>. Those rules are written
with some basic blocks that belong to <em>Language Theory</em> – if you’ve ever heard of Stephen Kleene and
its famous <a href="https://en.wikipedia.org/wiki/Kleene_star">Kleene star</a> for instance, you will feel familiar with PEG.</p>
<p>What I love about PEG is that with a very limited set of constructs, you can describe a lot of
determinist languages. In the case of GLSL450 – which is the language that the <a href="https://crates.io/crates/glsl">glsl</a> crate can
parse – it’s a <em>context-free</em> and <em>determinist</em> language. So the whole language can be defined in
terms of (recursive) PEG rules.</p>
<p>The PEG primitive constructs available in <a href="https://crates.io/crates/pest">pest</a> are:</p>
<ul>
<li>Matching against a lexeme, like <code>&quot;foo&quot;</code>: this rule will recognize any string that is <code>&quot;foo&quot;</code>.</li>
<li>Matching against a range of char, like <code>'a' .. 'z'</code>: this will recognize any char in the range,
like <code>&quot;a&quot;</code>, <code>&quot;d&quot;</code> or <code>&quot;z&quot;</code>.</li>
<li>Matching against another rule, simply by using its name.</li>
<li>Matching a sequence of rules, like <code>a ~ b</code>: this rule will be matched if it can match the <code>a</code>
rule followed by the <code>b</code> rule; <code>&quot;foo&quot; ~ 'a' .. 'z'</code> will recognize <code>&quot;foot&quot;</code>, <code>&quot;food&quot;</code>, <code>&quot;fooz&quot;</code>
but also <code>&quot;foo r&quot;</code> – this whitespace behavior is explained below – etc.</li>
<li>Matching either a rule or another (alternative), like <code>a | b</code>: this rule will match either <code>a</code>
if it succeeds or <code>b</code> if <code>a</code> fails and <code>b</code> succeeds or none otherwise. <code>&quot;foo&quot; | 'a' .. 'z'</code> will
be matched with <code>&quot;foo&quot;</code> and <code>&quot;a&quot;</code> or <code>&quot;f&quot;</code>.</li>
<li>Optional matching with the <code>?</code> suffix operator, like <code>a?</code>: this will match <code>a</code> if it succeeds or
will just ignore its failure if it fails, making it optional. For instance, <code>&quot;hello&quot;? ~ &quot;world&quot;</code>
will match both <code>&quot;hello world&quot;</code> and <code>&quot;world&quot;</code>.</li>
<li>Parenthesis can be used to group rules and reason on them as a whole, like <code>(a ~ b)</code>.</li>
<li>Repetitions:
<ul>
<li>The <a href="https://en.wikipedia.org/wiki/Kleene_star">Kleene star</a> (matches zero or many), like <code>a*</code>: this rule matches zero or more times the
<code>a</code> rule. For instance, <code>('0' .. '9')*</code> will match <code>&quot;0&quot;</code> and <code>&quot;314&quot;</code> but will also match <code>&quot;&quot;</code>.</li>
<li>The <a href="https://en.wikipedia.org/wiki/Kleene_star#Kleene_plus">Kleene plus</a> (matches one or many), like <code>a+</code>: this rule matches one or more times the
<code>a</code> rule. For instance, <code>('0' .. '9')+</code> will match <code>&quot;0&quot;</code> and <code>&quot;314&quot;</code> but will not match <code>&quot;&quot;</code>
because it expects at least one digit.</li>
<li>The <code>{min, max}</code> notation. The rule <code>a{0,}</code> is akin to <code>a*</code>: it must be matched at least 0
times and at maximum infinite times. The rule <code>a{,3}</code> must be matched between 0 and 3
times and the rule <code>a{1, 12}</code> must be matched between 1 and 12 times.</li>
</ul>
</li>
</ul>
<p>Additionally, <a href="https://crates.io/crates/pest">pest</a> provides a notation to add <em>rule modifiers</em>:</p>
<ul>
<li>A simple rule is written <code>rule_name { definition }</code>, like <code>number = { ASCII_DIGIT+ }</code>.</li>
<li><code>~</code>, <code>*</code> and <code>+</code> rules get interleaved with two special rules <strong>if defined</strong>:
<ul>
<li><code>WHITESPACE</code>, defining how to eat a single whitespace.</li>
<li><code>COMMENT</code>, defining how to eat a comment.</li>
<li>Those rules are interleaved as <code>(WHITESPACE | COMMENT)*</code> so that <code>number = { ASCII_DIGIT+ }</code>
gets actually rewritten as <code>number = { (ASCII_DIGIT | (WHITESPACE | COMMENT)*)+ }</code>.</li>
</ul>
</li>
<li>Because of that last rule, sometimes we don’t want that to happen. In the case of the <code>number</code>
rule defined above, both <code>&quot;324&quot;</code> and <code>&quot;3 2 4&quot;</code> will get matched if we defined <code>WHITESPACE</code> to
eat <code>&quot; &quot;</code>. In order to fix this, the <code>@</code> modifier can be prepended to the rule definition:
<ul>
<li><code>number = @{ ASCII_DIGIT+ }</code></li>
<li>This rule will not get interleaved at all and is <em>atomic</em>.</li>
<li>Easy-to-remember: <code>@</code> for <em>@tomic</em>. :)</li>
</ul>
</li>
<li>The <code>_</code> modifier can be used to mute a rule. For instance, eating whitespaces will not provide
you with any useful tokens. That’s why the <code>WHITESPACE</code> rule is defined as
<code>WHITESPACE = _{ &quot; &quot; }</code>.</li>
<li>There are some other modifiers, like <code>$</code> and <code>!</code> but we’re not interested in them as they’re
very specific.</li>
</ul>
<p>So, basically, you write a (long) list of PEG rules in a file, give that file to Rust via a
proc-macro derive, <em>et voila</em>!</p>
<pre><code>#[derive(Parser)]
#[grammar = &quot;path/to/grammar.pest&quot;]
struct Parser;
</code></pre>
<p>It’s that simple.</p>
<h1>Facing issues</h1>
<p>PEG is really cool to use… but there are caveats. For instance, PEG / <a href="https://crates.io/crates/pest">pest</a> cannot express
<em>left-recursive</em> rules. To understand this, take this rule:</p>
<pre><code>foo = { a | b ~ foo* }
</code></pre>
<p>This rule is correct and will allow you to match against <code>&quot;a&quot;</code> or <code>&quot;b&quot;</code> or even <code>&quot;bbbbbbbbb&quot;</code> and
<code>&quot;bbbbbbbba&quot;</code>. However, the following one is left-recursive and won’t be able to be compiled:</p>
<pre><code>foo = { a | foo* ~ b }
</code></pre>
<p>As you can see, the second alternative is left-recursive. Imagine matching against <code>&quot;a&quot;</code>: the first
alternative fires, everything is great. Now match against <code>&quot;b&quot;</code>. The first alternative fails so
<a href="https://crates.io/crates/pest">pest</a> tries the second one, but in order to take the second one, it needs to try at least once
<code>foo</code> (the <code>foo*</code>) to see whether it fails or not. If it fails, it will then ignore and try to eat a
<code>&quot;b&quot;</code>. If it succeeds, it will try to fire the <code>foo</code> rule again. However, when it tries <code>foo</code> for
the first time, it will try the first alternative again, <code>a</code>, then will fail, then will try <code>foo*</code>,
then <code>a</code>, then <code>foo*</code>, then… You see the pattern.</p>
<p>Left-recursive rules are then forbidden so that we can escape that situation. PEGs need to be
unambiguous and the rules (especially alternatives) are tested <strong>in order</strong>. You will have to unroll
and optimize those rules by hand. In our case, we can rewrite that rule in a PEG-compliant way:</p>
<pre><code>foo = { (a | b)+ }
</code></pre>
<p>An issue I had while writing the PEG of GLSL450 was with the <code>expression</code> rule (among a loooot of
others). Khronos’ specifications are written with left-recursive rules pretty much everywhere and
this is the one for expressions:</p>
<pre><code>expression:
  assignment_expression
  expression COMMA assignment_expression
</code></pre>
<p>Read it as <em>“An expression is either an assignment expression or an expression with a comma followed
by an assignment expression”</em>. This actually means that the second alternative cannot be constructed
without the first one – because it’s recursive! So you must start with an <code>assignment_expression</code>.
Then you can have the second rule applied recursively, like so (parenthesis just for convenience to
show the left-recursion):</p>
<pre><code>((((assignment_expression) COMMA assignment_expression) COMMA assignment_expression) COMMA assignment_expression)
</code></pre>
<p>Etc., etc. If you remove the parenthesis, you get:</p>
<pre><code>assignment_expression COMMA assignment_expression COMMA assignment_expression COMMA assignment_expression
</code></pre>
<p>Which you should now be able to write with a PEG rule easily! Like so:</p>
<pre><code>expression = { assignment_expression ~ (COMMA ~ assignment_expression)* }
</code></pre>
<p>Way better and without left-recursion! :) So I spent pretty much a whole afternoon trying to remove
left-recursions (and some rules were really nasty). I even have one remaining rule I’ll have to cut
like a surgeon later to make it fully available (I had to do the same thing with <a href="https://crates.io/crates/nom">nom</a>; it’s the
rule allowing <em>postfix expressions</em> to be used as <em>function identifiers</em>; issue
<a href="https://github.com/phaazon/glsl/issues/45">here</a>).</p>
<h1>Is it really a parser?</h1>
<p>Then I started to play with those generated <code>Rule</code> types… wait, what?! <em>The</em> generated <code>Rule</code> type?
Ok so that was my first surprise: the grammar knows how the rules are (deeply) nested but <a href="https://crates.io/crates/pest">pest</a>
outputs a single <code>Rule</code> type with all the rules laid down as flat variants for that <code>Rule</code>! This was
at first very confusing for me as I thought I’d be visiting freely through the <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">AST</a> – because this
is what a grammar actually defines. Instead, you must ask your <code>Parser</code> type which rule you want
to match against. You get a <code>Pairs</code>, which is an iterator over <code>Pair</code>s.</p>
<p>A <code>Pair</code> defines a single token that is defined by its starting point and ending point (hence a
pair… yeah I know, it’s confusing). A <code>Pair</code> has some methods you can call on it, like <code>as_str()</code> to
retrieve its string slice in the (borrowed) input of your parser. Or <code>into_inner()</code>, giving you a
<code>Pairs</code> that represent the matched inner tokens, allowing you to eventually visit through the tree
of tokens.</p>
<p>I have two problems with that.</p>
<h2>My unreachable goals</h2>
<p>First, because <a href="https://crates.io/crates/pest">pest</a> doesn’t fully use the grammar information <em>at type-level</em>, you lose the one
crucial information: the actual topology of your grammar in the type system. That means that every
time you will ask for a given <code>Pair</code>, you will have to call <code>as_rule()</code> on it an pattern-match on
<strong>the whole list of variants that compose your grammar!</strong> In all situations, you will only need to
match against a very few of them and will discard every others. And how do you discard them? –
you <em>know</em> those are impossible because the very definition of the grammar? You use
<code>unreachable!()</code>, which I find almost as unsafe and ugly as using <code>unimplemented!()</code>. I know
<code>unreachable!()</code> is required in certain situations, but having it spawn in a parser using the public
API of <a href="https://crates.io/crates/pest">pest</a> makes me gnash my teeth. Example of code I’m currently writing:</p>
<pre><code>fn parse_type_specifier_non_array&lt;'a&gt;(pair: Pair&lt;'a&gt;) -&gt; Result&lt;syntax::TypeSpecifierNonArray, String&gt; {
  let inner = pair.into_inner().next().unwrap();

  match inner.as_rule() {
    Rule::struct_specifier =&gt; {
      // …
    }

    Rule::identifier =&gt; {
      let identifier = inner.as_str();

      match identifier {
        &quot;void&quot; =&gt; Ok(syntax::TypeSpecifierNonArray::Void),
        &quot;float&quot; =&gt; Ok(syntax::TypeSpecifierNonArray::Float),
        &quot;double&quot; =&gt; Ok(syntax::TypeSpecifierNonArray::Double),
        // …
        _ =&gt; Ok(syntax::TypeSpecifierNonArary::TypeName(identifier.to_owned())
      }
    }

    _ =&gt; unreachable!()
  }
}
</code></pre>
<p>And I reckon I will find that pattern a lot in my next few hours / days of writing those functions.</p>
<h2>Lexing and parsing are two different concepts</h2>
<p>That <code>parse_type_specifier_non_array</code> function of mine also pinpointed another issue I had: initially,
I wrote that logic in the <code>type_specifier_non_array</code> rule in the PEG grammar, like:</p>
<pre><code>type_specifier_non_array = {
  &quot;void&quot; |
  &quot;float&quot; |
  // …
  identifier
}
</code></pre>
<p>And quickly came to the realization that the <code>identifier</code> alternative would get <em>muted</em> very often
by the lexemes alternatives. Imagine two strings, <code>&quot;int x = 3;&quot;</code> and <code>&quot;interop x = …;&quot;</code>. The first
type will get the right <code>TypeSpecifierNonArray::Int</code> AST representation and the second string… will
get the same because the <code>&quot;int&quot;</code> rule will fire first, leaving some unmatched text without any
sense (<code>&quot;erop x = …;&quot;</code>).</p>
<p>What that means is that we cannot express this rule this way with PEG. I’ve been told <a href="https://crates.io/crates/pest">pest</a> should
now support the <code>!</code> operator that can be used to encode something that <em>must not be parsed</em>. But
that would just slightly hide the problem: there is no way in the PEG rules to actually do parsing
transformation. What <a href="https://crates.io/crates/pest">pest</a> really is here is a <em>lexer</em>: it generates tokens. Yes, it does place
them in a tree (token tree), but they all remains tokens (strings). And that is when I thought about
the difference between <a href="https://crates.io/crates/pest">pest</a> and <a href="https://crates.io/crates/nom">nom</a></p>
<h1>Pest, nom… let’s explain</h1>
<p><a href="https://crates.io/crates/nom">nom</a> is a parser combinator. That means you can build bigger parsers by combining small parsers.
The correct term to talk about <a href="https://crates.io/crates/nom">nom</a> is that it’s a <a href="https://en.wikipedia.org/wiki/Scannerless_parsing">scannerless parser</a>: it doesn’t require to
generate tokens prior to do parsing and prefers to do <strong>both at the same time</strong>. <a href="https://crates.io/crates/nom">nom</a> parsers are
typically implemented using macros like <code>preceded!</code>, <code>delimited!</code>, <code>take_until!</code>, <code>tag!</code>, <code>value!</code>
and <code>do_parse!</code>, allowing for matching (lexing) slices of bytes / chars <em>and</em> parsing them to actual
values with the type of your choice.</p>
<p>However, <a href="https://crates.io/crates/pest">pest</a> relies on a PEG file, representing the formal grammar of a language to tokenize.
That lexer phase takes place and has to be able to tokenize the whole input before giving hand back.
I’m not sure when I say this (but I’m pretty convince it’s the case): <a href="https://crates.io/crates/pest">pest</a> doesn’t support
streaming input, since it needs to eat the special rule <code>EOI</code> – End Of Input – or eat a rule error
(to succeed with the previous rule or propagate the error upwards) before returning. <a href="https://crates.io/crates/nom">nom</a> can be
used to eat streams of bytes, though.</p>
<p>And then, you have two choices with <a href="https://crates.io/crates/pest">pest</a>:</p>
<ol>
<li>Either you decide to expose the tokens as a token tree, in which case the AST contains string
slices, completely left to be parsed.</li>
<li>Either, like me, you have your own AST representation and you will have to write the actual
parser over <a href="https://crates.io/crates/pest">pest</a>.</li>
</ol>
<p>I’m currently experimenting with (2.). The <code>type_specifier_non_array</code> rule is one of the biggest one
in my grammar so maybe I’ve already done the worst case but I’m fearing I’ll have long nights of
coding before ending up with a benchmark <a href="https://crates.io/crates/pest">pest</a> vs. <a href="https://crates.io/crates/nom">nom</a>.</p>
<h1>Call to contributions</h1>
<p>In the meantime:</p>
<ul>
<li>I’ve written a very simple SPIR-V transpiler (that uses <a href="https://crates.io/crates/shaderc">shaderc</a> behind the hood). If you’re
looking for an easy contribution, I’m looking for someone to test the feature branch holding
the implementation, cleaning it, writing tests for it and merging it to <code>master</code>.
<a href="https://github.com/phaazon/glsl/issues/21">Issue here</a>.</li>
<li>I’m looking for a way to encode <code>#define</code> and other preprocessors pragmas. I’m taking
contributions (implementations, ideas) to fix that problem.
<a href="https://github.com/phaazon/glsl/issues/39">Issue here</a>.</li>
<li>If you want to start working on refactoring the AST a bit (it was designed based on the Khronos
grammar rules… which are quite ugly :P), please feel free to open an issue. I will eventually
do it anyway sooner or later, but you know… :)</li>
<li>A lot of the types in the <code>syntax</code> module (the AST types) don’t have <code>new</code> methods. Those
methods are important to me as they enable to bypass some limitation from the current
<code>glsl-quasiquote</code> implementation that doesn’t allow variable interpolation.</li>
<li>Anything else! Feel free to contribute!</li>
</ul>
<p>Thanks for having read through, and keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/lifetimes_and_self_borrowingTue, 07 Feb 2017 00:00:00 GMT
title: Lifetimes limits – self borrowing and dropchecker
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/lifetimes_and_self_borrowing
feedurl: https://phaazon.net/blog/feed
content:
<p>Lately, I’ve been playing around with <a href="https://crates.io/crates/alto">alto</a> in my demoscene framework. This crate in the <em>replacement of <a href="https://crates.io/crates/openal-rs">openal-rs</a></em> as <strong>openal-rs</strong> has
been deprecated because <em>unsound</em>. It’s a wrapper over <a href="https://www.openal.org">OpenAL</a>, which enables you to play 3D sounds and gives you several physical properties and effects you can apply.</p>
<h1>The problem</h1>
<p>Just to let you fully understand the problem, let me introduce a few principles from alto. As a wrapper over OpenAL, it exposes quite the same interface, but adds several safe-related types. In order to use
the API, you need three objects:</p>
<ul>
<li>an <code>Alto</code> object, which represents the <em>API object</em> (it holds dynamic library handles, function pointers, etc. ; we don’t need to know about that)</li>
<li>a <code>Device</code> object, a regular device (a sound card, for example)</li>
<li>a <code>Context</code> object, used to create audio resources, handle the audio context, etc.</li>
</ul>
<p>There are well-defined relationships between those objects that state about their lifetimes. An <code>Alto</code> object must outlive the <code>Device</code> and the <code>Device</code> must outlive the <code>Context</code>.
Basically:</p>
<pre><code class="language-rust">let alto = Alto::load_default(None).unwrap(); // bring the default OpenAL implementation in
let dev = alto.open(None).unwrap(); // open the default device
let ctx = dev.new_context(None).unwrap(); // create a default context with no OpenAL extension
</code></pre>
<p>As you can see here, the lifetimes are not violated, because <code>alto</code> outlives <code>dev</code> which outlives <code>ctx</code>. Let’s dig in the type and function signatures to get the lifetimes right
(documentation <a href="https://docs.rs/alto/1.0.5/alto">here</a>).</p>
<pre><code class="language-rust">fn Alto::open&lt;'s, S: Into&lt;Option&lt;&amp;'s CStr&gt;&gt;&gt;(&amp;self, spec: S) -&gt; AltoResult&lt;Device&gt;
</code></pre>
<p>The <code>S</code> type is just a convenient type to select a specific implementation. We need the default one, so just pass <code>None</code>. However, have a look at the result. <code>AltoResult&lt;Device&gt;</code>. I told you about
lifetime relationships. This one might be tricky, but you always have to wonder “is there an elided lifetime here?”. Look at the <code>Device</code> type:</p>
<pre><code class="language-rust">pub struct Device&lt;'a&gt; { /* fields omitted */ }
</code></pre>
<p>Yep! So, what’s the lifetime of the <code>Device</code> in <code>AltoResult&lt;Device&gt;</code>? Well, that’s simple: the lifetime elision rule in action is one of the simplest:</p>
<blockquote>
<p>If there are multiple input lifetime positions, but one of them is &amp;self or &amp;mut self, the lifetime of self is assigned to all elided output lifetimes.
(<a href="https://doc.rust-lang.org/beta/nomicon/lifetime-elision.html">source</a>)</p>
</blockquote>
<p>So let’s rewrite the <code>Alto::open</code> function to make it clearer:</p>
<pre><code class="language-rust">fn Alto::open&lt;'a, 's, S: Into&lt;Option&lt;&amp;'s CStr&gt;&gt;&gt;(&amp;'a self, spec: S) -&gt; AltoResult&lt;Device&lt;'a&gt;&gt; // exact same thing as above
</code></pre>
<p>So, what you can see here is that the <code>Device</code> must be valid for the same lifetime as the reference we pass in. Which means that <code>Device</code> cannot outlive the reference. Hence, it cannot outlive the
<code>Alto</code> object.</p>
<hr />
<pre><code class="language-rust">impl&lt;'a&gt; Device&lt;'a&gt; {
  // …
  fn new_context&lt;A: Into&lt;Option&lt;ContextAttrs&gt;&gt;&gt;(&amp;self, attrs: A) -&gt; AltoResult&lt;Context&gt;
  // …
}
</code></pre>
<p>That looks a bit similar. Let’s have a look at <code>Context</code>:</p>
<pre><code class="language-rust">pub struct Context&lt;'d&gt; { /* fields omitted */ }
</code></pre>
<p>Yep, same thing! Let’s rewrite the whole thing:</p>
<pre><code class="language-rust">impl&lt;'a&gt; Device&lt;'a&gt; {
  // …
  fn new_context&lt;'b, A: Into&lt;Option&lt;ContextAttrs&gt;&gt;&gt;(&amp;'b self, attrs: A) -&gt; AltoResult&lt;Context&lt;'b&gt;&gt;
  // …
}
</code></pre>
<p>Plus, keep in mind that <code>self</code> is actually <code>Device&lt;'a&gt;</code>. The first argument of this function then awaits a <code>&amp;'b Device&lt;'a&gt;</code> object!</p>
<blockquote>
<p>rustc is smart enough to automatically insert the <code>'a: 'b</code> lifetime bound here – i.e. the 'a lifetime outlives 'b. Which makes sense: the reference will die before the <code>Device&lt;'a&gt;</code> is dropped.</p>
</blockquote>
<p>Ok, ok. So, what’s the problem then?!</p>
<h1>The (real) problem</h1>
<p>The snippet of code above about how to create the three objects is straight-forward (though we don’t take into account errors, but that’s another topic). However, in my demoscene framework, I really don’t
want people to use that kind of types. The framework should be completely agnostic about which technology or API is used internally. For my purposes, I just need a single type with a few methods to work
with.</p>
<p>Something like that:</p>
<pre><code class="language-rust">struct Audio = {}

impl Audio {
  pub fn new&lt;P&gt;(track_path: P) -&gt; Result&lt;Self&gt; where P: AsRef&lt;Path&gt; {}

  pub fn toggle(&amp;mut self) -&gt; bool {}

  pub fn playback_cursor(&amp;self) -&gt; f32 {}

  pub fn set_playback_cursor(&amp;self, t: f32) {}
}

impl Drop for Audio {
  fn drop(&amp;mut self) {
    // stop the music if playing; do additional audio cleanup
  }
}
</code></pre>
<p>This is a very simple interface, yet I don’t need more. <code>Audio::set_playback_cursor</code> is cool when I debug my demos in realtime by clicking a time panel to quickly jump to a part of the music.
<code>Audio::toggle()</code> enables me to pause the demo to inspect an effect in the demo. Etc.</p>
<p>However, how can I implement <code>Audio::new</code>?</p>
<h1>The (current) limits of borrowing</h1>
<p>The problem kicks in as we need to wrap the three types – <code>Alto</code>, <code>Device</code> and <code>Context</code> – as the fields of <code>Audio</code>:</p>
<pre><code class="language-rust">struct Audio&lt;'a&gt; {
  alto: Alto,
  dev: Device&lt;'a&gt;,
  context: Context&lt;'a&gt;
}
</code></pre>
<p>We have a problem if we do this. Even though the type is correct, we cannot correctly implement <code>Audio::new</code>. Let’s try:</p>
<pre><code class="language-rust">impl&lt;'a&gt; Audio&lt;'a&gt; {
  pub fn new&lt;P&gt;(_: P) -&gt; Result&lt;Self&gt; where P: AsRef&lt;Path&gt; {
    let alto = Alto::load_default(None).unwrap();
    let dev = alto.open(None).unwrap();
    let ctx = dev.new_context(None).unwrap();

    Ok(Audio {
      alto: alto,
      dev: dev,
      ctx: ctx
    })
  }
}
</code></pre>
<p>As you can see, that cannot work:</p>
<pre><code>error: `alto` does not live long enough
  --&gt; /tmp/alto/src/main.rs:14:15
   |
14 |     let dev = alto.open(None).unwrap();
   |               ^^^^ does not live long enough
...
22 |   }
   |   - borrowed value only lives until here
   |
note: borrowed value must be valid for the lifetime 'a as defined on the body at 12:19...
  --&gt; /tmp/alto/src/main.rs:12:20
   |
12 |   fn new() -&gt; Self {
   |                    ^

error: `dev` does not live long enough
  --&gt; /tmp/alto/src/main.rs:15:15
   |
15 |     let ctx = dev.new_context(None).unwrap();
   |               ^^^ does not live long enough
...
22 |   }
   |   - borrowed value only lives until here
   |
note: borrowed value must be valid for the lifetime 'a as defined on the body at 12:19...
  --&gt; /tmp/alto/src/main.rs:12:20
   |
12 |   fn new() -&gt; Self {
   |                    ^

error: aborting due to 2 previous errors
</code></pre>
<p>What’s going on here? Well, we’re hitting a problem called the problem of <strong>self-borrowing</strong>. Look at the first two lines of our implementation of <code>Audio::new</code>:</p>
<pre><code class="language-rust">let alto = Alto::load_default(None).unwrap();
let dev = alto.open(None).unwrap();
</code></pre>
<p>As you can see, the call to <code>Alto::open</code> borrows <code>alto</code> – via a <code>&amp;Alto</code> reference. And of course, you cannot move a value that is borrowed – that would invalidate all the references pointing to it.
We also have another problem: imagine we could do that. All those types implement <code>Drop</code>. Because they basically all have the same lifetime, there’s no way to know which one borrows information from whom.
The <em>dropchecker</em> has no way to know that. It will then refuse to code creating objects of this type, because dropping might be unsafe in that case.</p>
<h1>What can we do about it?</h1>
<p>Currently, this problem is linked to the fact that the lifetime system is a bit too restrictive and doesn’t allow for <strong>self-borrowing</strong>. Plus, you also have the <em>dropchecker</em> issue to figure out. Even
though we were able to bring in <code>alto</code> and <code>device</code> altogether, how do you handle <code>context</code>? The <em>dropchecker</em> doesn’t know which one must be dropped first – there’s no obvious link at this stage between
<code>alto</code> and all the others anymore, because that link was made with a reference to <code>alto</code> that died – we’re moving out of the scope of the <code>Audio::new</code> function.</p>
<p><img src="http://phaazon.net/pub/rust_self_borrowing.jpg" alt="" /></p>
<p>That’s a bit tough. The current solution I implemented to fix the issue is ok–ish, but I dislike it because it adds a significant performance overhead: I just moved the initialization code in a thread that
stays awake until the <code>Audio</code> object dies, and I use a synchronized channel to communicate with the objects in that thread. That works because the thread provides us with a stack, that is the support of
lifetimes – think of scopes.</p>
<p>Another solution would be to move that initialization code in a function that would accept a closure – your application. Once everything is initialized, the closure is called with a few callbacks to
toggle / set the cursor of the object living “behind” on the stack. I don’t like that solution because it modifies the main design – having an <code>Audio</code> object was the goal.</p>
<p>Other solutions are:</p>
<ul>
<li><code>std::mem::transmute</code> to remove the lifetimes (replace them with <code>'static</code>). That’s <strong>hyper dangerous</strong> and we are just breaking Rust’s lifetimes… <em>not okay</em> :(</li>
<li>change our design to meet the same as alto’s (in a word: use the same three objects)</li>
<li>cry deeply</li>
</ul>
<p>I don’t have a satisfying solution yet to that problem. My thread solution works and lets me have a single type abstracting all of that, but having a thread for such a thing is a waste of resources to me. I
think I’ll implement the closure solution as, currently, it’s not possible to embed in struct lifetimes’ semantics / logic. I guess it’s okay; I guess the problem is also linked to the fact the concept is
pretty young and we’re still kind of experimenting it. But clearly, lifetimes hit a hard problem here that they cannot solve correctly. Keep in mind that even if unsafe solutions exist, we’re talking about
a library that’s designed to work with Rust lifetimes as a pretty high level of abstraction. Firing <code>transmute</code> is very symptomatic of something wrong. I’m open to suggestions, because I’ve been thinking the
problem all day long without finding a proper solution.</p>
<p>Keep the vibe!</p>


guid: https://strongly-typed-thoughts.net/blog/luminance-0.6Sun, 25 Oct 2015 00:00:00 GMT
title: luminance, episode 0.6: UBO, SSBO, Stackage
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance-0.6
feedurl: https://phaazon.net/blog/feed
content:
<p>Up to now, <a href="https://hackage.haskell.org/package/luminance">luminance</a> has been lacking two cool
features: <a href="https://www.opengl.org/wiki/Uniform_Buffer_Object">UBO</a> and
<a href="https://www.opengl.org/wiki/Shader_Storage_Buffer_Object">SSBO</a>. Both are <em>buffer-backed</em> uniform
techniques. That is, a way to pass uniforms to shader stages through buffers.</p>
<p>The <a href="https://hackage.haskell.org/package/luminance-0.6">latest version of luminance</a> has one of the
two features. <em>UBO</em> were added and <em>SSBO</em> will follow for the next version, I guess.</p>
<h1>What is UBO?</h1>
<p>UBO stands for <strong>U</strong>niform <strong>B</strong>buffer <strong>O</strong>bject. Basically, it enables you to create <em>uniform
blocks</em> in GLSL in feed them with <em>buffers</em>. Instead of passing values directly to the uniform
interface, you just write whatever values you want to to buffers, and then pass the buffer as a
source for the uniform block.</p>
<p>Such a technique has a lot of advantages. Among them, you can pass <strong>a lot of values</strong>. It’s also
cool when you want to pass values instances of a structure (in the GLSL source code). You can also
use them to share uniforms between several shader programs as well as quickly change all the
uniforms to use.</p>
<p>In luminance, you need several things. First thing first, you need… a buffer! More specifically,
you need a buffer <a href="https://hackage.haskell.org/package/luminance-0.6/docs/Graphics-Luminance-Buffer.html#t:Region"><code>Region</code></a>
to store values in. However, you cannot use any kind of region. You have to use a region that can
hold values that will be fetched from shaders. This is done with a type called <code>UB a</code>. A buffer of
<code>UB a</code> can be used as <em>UBO</em>.</p>
<p>Let’s say you want to store colors in a buffer, so that you can use them in your fragment shader.
We’ll want three colors to shade a triangle. We need to create the buffer and get the region:</p>
<pre><code class="language-haskell">colorBuffer :: Region RW (UB (V3 Float)) &lt;- createBuffer (newRegion 3)
</code></pre>
<p>The explicit type is there so that GHC can infer the correct types for the <code>Region</code>. As you can see,
nothing fancy, except that we just don’t want a <code>Region RW (V3 Float</code> but
<code>Region RW (UB (V3 Float))</code>. Why <code>RW</code>?</p>
<p>Then, we’ll want to store colors in the buffer. Easy peasy:</p>
<pre><code class="language-haskell">writeWhole colorBuffer (map UB colors)

colors :: [V3 Float]
colors = [V3 1 0 0,V3 0 1 0,V3 0 0 1] -- red, green, blue
</code></pre>
<p>At this point, <code>colorBuffer</code> represents a GPU buffer that holds three colors: red, green and blue.
The next part is to get the uniform interface. That part is experimental in terms of exposed
interface, but the core idea will remain the same. You’re given a function to build <em>UBO</em> uniforms
as you also have a function to build simple and plain uniforms in
<a href="https://hackage.haskell.org/package/luminance-0.6/docs/Graphics-Luminance-Shader-Program.html#v:createProgram"><code>createProgram</code></a>:</p>
<pre><code class="language-haskell">createProgram shaderList $ \uni uniBlock -&gt; {- … -}
</code></pre>
<p>Don’t spend too much time reading the signature of that function. You just have to know that
<code>uni</code> is a function that takes <code>Either String Natural</code> – either a uniform’s name or its integral
semantic – and gives you mapped <code>U</code> in return and that <code>uniBlock</code> does the same thing, but for
uniform blocks instead.</p>
<p>Here’s our vertex shader:</p>
<pre><code class="language-C">in vec2 co;
out vec4 vertexColor;

// This is the uniform block, called &quot;Colors&quot; and storing three colors
// as an array of three vec3 (RGB).
uniform Colors {
  vec3 colors[3];
};

void main() {
  gl_Position = vec4(co, 0., 1.);
  vertexColor = vec4(colors[gl_VertexID], 1.);
}&quot;
</code></pre>
<p>So we want to get a <code>U a</code> mapped to that <code>&quot;Colors&quot;</code> uniform block. Easy!</p>
<pre><code class="language-haskell">(program,colorsU) &lt;- createProgram shaderStages $ \_ uniBlock -&gt; uniBlock &quot;Colors&quot;
</code></pre>
<p>And that’s all! The type of <code>colorsU</code> is <code>U (Region rw (UB (V3 Float)))</code>. You can then gather 
<code>colorBuffer</code> and <code>colorsU</code> in a uniform interface to send <code>colorBuffer</code> to <code>colorsU</code>!</p>
<p>You can find the complete sample <a href="https://github.com/phaazon/luminance-samples/blob/c4f428dc68ed38f0b80032aa9de90df01bf8ab15/src/UBO.hs">here</a>.</p>
<p>Finally, you can augment the type you can use <code>UB</code> with by implementing the <code>UniformBlock</code>
typeclass. You can derive the <code>Generic</code> typeclass and then use a default instance:</p>
<pre><code class="language-haskell">data MyType = {- … -} deriving (Generic)

instance UniformBlock MyTpe -- we’re good to go with buffer of MyType!
</code></pre>
<h1>luminance, luminance-samples and Stackage</h1>
<p>I added <em>luminance</em> and <em>luminance-samples</em> into Stackage. You can then find them in the nightly
snapshots and the future LTS ones.</p>
<h1>What’s next?</h1>
<p>I plan to add stencil support for the framebuffer, because it’s missing and people might like it
included. I will of course add support for <em>SSBO</em>* as soon as I can. I also need to work on cheddar
but that project is complex and I’m still stuck with design decisions.</p>
<p>Thanks for reading my and for your feedback. Have you great week!</p>


title: luminance-0.5.1 and wavefront-0.4.0.1
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance-0.5.1_and_wavefront-0.4.0.1
feedurl: https://phaazon.net/blog/feed
guid:
https://strongly-typed-thoughts.net/blog/luminance-0.5.1_and_wavefront-0.4.0.1Sun, 18 Oct 2015 00:00:00 GMT

content:
<p>It’s been a few days I haven’t talked about
<a href="http://hackage.haskell.org/package/luminance">luminance</a>. I’ve been working on it a lot those days
along with <a href="http://hackage.haskell.org/package/wavefront">wavefront</a>. In order that you keep up to
date, I’ll describe the changes I made in those packages you have a talk about the future directions
of those packages.</p>
<p>I’ll also give a snippet you can use to load geometries with wavefront and adapt them to embed into
luminance so that you can actually render them! A package might come up from that kind of snippet –
<code>luminance-wavefront</code>? We’ll see that!</p>
<h1>wavefront</h1>
<p>This package has received several changes among two major increments and several fixes. In the first
place, I removed some code from the interface that was useless and used only for test purposes. I
removed the <code>Ctxt</code> object – it’s a type used by the internal lexer anyways, so you don’t have to
know about it – and exposed a type called <a href="http://hackage.haskell.org/package/wavefront-0.4.0.1/docs/Codec-Wavefront.html#t:WavefrontOBJ">WavefrontOBJ</a>.
That type reprents the parsed Wavefront data and is the <em>main</em> type used by the library in the
interface.</p>
<p>Then, I also removed most of the modules, because they’re re-exported by the main module –
<a href="http://hackage.haskell.org/package/wavefront-0.4.0.1/docs/Codec-Wavefront.html">Codec.Wavefront</a>.
I think the documentation is pretty straight-forward, but you think something is missing, please
shoot a PM or an email! ;)</p>
<p>On the bugs level, I fixed a few things. Among them, there was a nasty bug in the implementation of
an internal recursive parser that caused the last wavefront statement to be silently ignored.</p>
<p>I’d also like to point out that I performed some benchmarks – I will provide the data later on with
a heap profile and graphs – and I’m pretty astonished with the results! The parser/lexer is insanely
fast! It only takes a few milliseconds (between 7ms and 8ms) to load 50k faces (a 2MB .obj file).
The code is not yet optimized, so I guess the package could go even faster!</p>
<p>You can find the changelog <a href="http://hackage.haskell.org/package/wavefront-0.4.0.1/changelog">here</a>.</p>
<h1>luminance</h1>
<p>I made a <em>lot of work</em> on luminance lately. First, the <code>V</code> type – used to represent <em>vertex
components</em> – is not anymore defined by luminance but by <a href="http://hackage.haskell.org/package/linear">linear</a>.
You can find the type <a href="http://hackage.haskell.org/package/linear-1.20.2/docs/Linear-V.html">here</a>.
You’ll need the <code>DataKinds</code> extension to write types like <code>V 3 Float</code>.</p>
<p>That change is due to the fact linear is a mature library with a lot of interesting functions and
types everyone might use when doing graphics. Its <a href="http://hackage.haskell.org/package/linear-1.20.2/docs/Linear-V.html"><code>V</code></a>
type has several interesting instances – <code>Storable</code>, <code>Ord</code>, etc. – that are required in luminance.
Because it’s not simple to build such <code>V</code>, luminance provides you with three functions to build the
1D, 2D and 3D versions – <a href="http://hackage.haskell.org/package/luminance-0.5.1/docs/Graphics-Luminance-Vertex.html#v:vec2"><code>vec2</code></a>,
<a href="http://hackage.haskell.org/package/luminance-0.5.1/docs/Graphics-Luminance-Vertex.html#v:vec3"><code>vec3</code></a>
and <a href="http://hackage.haskell.org/package/luminance-0.5.1/docs/Graphics-Luminance-Vertex.html#v:vec2"><code>vec4</code></a>.
Currently, that type is the only one you can use to build <em>vertex components</em>. I might add <code>V2</code>,
<code>V3</code> and <code>V4</code> as well later.</p>
<p>An interesting change: the <code>Uniform</code> typeclass has <strong>a lot</strong> of new instances! Basically, all vector
types from linear, their array version and the 4x4 floating matrix – <code>M44 Float</code>. You can find the
list of all instances <a href="http://hackage.haskell.org/package/luminance-0.5.1/docs/Graphics-Luminance-Shader-Uniform.html#t:Uniform">here</a>.</p>
<p>A new function was added to the <a href="http://hackage.haskell.org/package/luminance-0.5.1/docs/Graphics-Luminance-Geometry.html"><code>Graphics.Lumimance.Geometry</code></a>
module called <a href="http://hackage.haskell.org/package/luminance-0.5.1/docs/Graphics-Luminance-Geometry.html#v:nubDirect"><code>nubDirect</code></a>.
That function performs in linear logarithmic time and is used to turn a <em>direct</em> representation of
vertices into a pair of data used to represent <em>indexed</em> vertices. The new list of vertices
stores only unique vertices and the list of integral values stores the indices. You can then use
both the information to build <em>indexed geometries</em> – see
<a href="http://hackage.haskell.org/package/luminance-0.5.1/docs/Graphics-Luminance-Geometry.html#v:createGeometry"><code>createGeometry</code></a>
for further details.</p>
<p>The interface to transfer texels to textures has changed. It doesn’t depend on <code>Foldable</code> anymore
but on <code>Data.Vector.Storable.Vector</code>. That change is due to the fact that the <code>Foldable</code> solution
uses <code>toList</code> behind the hood, which causes bad performance for the simple reason that we send the
list to the GPU through the FFI. It’s then more efficient to use a <code>Storable</code> version. Furthermore,
th  most known package for textures loading – <a href="http://hackage.haskell.org/package/JuicyPixels">JuicyPixels</a>
– already uses that type of <code>Vector</code>. So you just have to enjoy the new performance boost! ;)</p>
<p>About bugs… I fixed a few ones. First, the implementation of the <code>Storable</code> instance for
<a href="http://hackage.haskell.org/package/luminance-0.5.1/docs/Graphics-Luminance-Core-Tuple.html#t::."><code>(:.)</code></a>
had an error for <code>sizeOf</code>. The implementation must be lazy in its argument, and the old one was not,
causing <code>undefined</code> crashes when using that type. The strictness was removed and now everything
works just fine!</p>
<p>Two bugs that were also fixed: the indexed render and the render of geometries with several vertex
components. Those bugs were easy to fix and now you won’t experience those issues anymore.</p>
<h1>Interfacing luminance with wavefront to render geometries from artists!</h1>
<p>I thought it would be a hard task but I’m pretty proud of how easy it was to interface both the
packages! The idea was to provide a function that would turn a <code>WavefrontOBJ</code> into a <em>direct
representation</em> of luminance vertices. Here’s the function that implements such a conversion:</p>
<pre><code class="language-haskell">type Vtx = V 3 Float :. V 3 Float -- location :. normal

objToDirect :: WavefrontOBJ -&gt; Maybe [Vtx]
objToDirect obj = traverse faceToVtx (toList faces)
  where
    locations = objLocations obj
    normals = objNormals obj
    faces = objFaces obj
    faceToVtx face = do
      let face' = elValue face
      vni &lt;- faceNorIndex face'
      v &lt;- locations !? (faceLocIndex face' - 1)
      vn &lt;- normals !? (vni - 1)
      let loc = vec3 (locX v) (locY v) (locZ v)
          nor = vec3 (norX vn) (norY vn) (norZ vn)
      pure (loc :. nor)
</code></pre>
<p>As you can see, that function is pure and will eventually turn a <code>WavefrontOBJ</code> into a list of
<code>Vtx</code>. <code>Vtx</code> is our own vertex type, encoding the location and the normal of the vertex. You can
add texture coordinates if you want to. The function fails if a face’s index has no normal
associated with or if an index is out-of-bound.</p>
<p>And… and that’s all! You can already have your <code>Geometry</code> with that – <em>direct</em> one:</p>
<pre><code class="language-haskell">  x &lt;- fmap (fmap objToDirect) (fromFile &quot;./ubercool-mesh.obj&quot;)
  case x of
    Right (Just vertices) -&gt; createGeometry vertices Nothing Triangle
    _ -&gt; throwError {- whatever you need as error there -}
</code></pre>
<p>You want an indexed version? Well, you already have everything to do that:</p>
<pre><code class="language-haskell">  x &lt;- fmap (fmap (nubDirect . objToDirect) (fromFile &quot;./ubercool-mesh.obj&quot;)
  case x of
    Right (Just (vertices,indices)) -&gt; createGeometry vertices (Just indices) Triangle
    _ -&gt; throwError {- whatever you need as error there -}
</code></pre>
<p>Even though the <code>nubDirect</code> performs in a pretty good complexity, it takes time. Don’t be surprised
to see the “loading” time longer then.</p>
<p>I might package those snippets and helpers around them into a <code>luminance-wavefront</code> package, but
that’s not trivial as the vertex format should be free.</p>
<h1>Future directions and thank you</h1>
<p>I received a lot of warm feedback from people about what I do in the Haskell community, and I’m just
amazed. I’d like to thank each and everyone of you for your support – I even got support from
non-Haskellers!</p>
<p>What’s next then… Well, I need to add a few more textures to luminance – texture arrays are not
supported yet, and the framebuffers have to be altered to support all kind of textures. I will also
try to write a <a href="">cheddar</a> interpreter directly into luminance to dump the <code>String</code> type of shader
stages and replace it with cheddar’s whatever will be. For the long terms, I’ll add UBO and SSBO to
luminance, and… compatibility with older OpenGL versions.</p>
<p>Once again, thank you, and keep the vibe!</p>


guid: https://strongly-typed-thoughts.net/blog/luminance_comparingWed, 28 Mar 2018 18:40:00 GMT
title: Main differences between luminance and other famous Rust crates
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance_comparing
feedurl: https://phaazon.net/blog/feed
content:
<p>It’s been a while since I’ve been wanting to do this. Either on IRC, Reddit, GitHub Issues, IRL or
other any other way of communicating, there are more and more people asking me to <em>compare
<a href="https://crates.io/crates/luminance">luminance</a> to other, well known and famous, rendering-like frameworks</em>.</p>
<p>I’ve made a listing with pros and cons. in the <a href="https://github.com/phaazon/luminance-rs#how-does-it-compare-to">README</a> file of <a href="https://crates.io/crates/luminance">luminance</a>. I’ll just copy that
section and report it below so that you’re not even a click away from the listing!</p>
<h1>How does it compare to…</h1>
<p>A lot of folks have asked me on IRC / Reddit / gitter / etc. how <a href="https://crates.io/crates/luminance">luminance</a> compares to other
famous frameworks. Here are a few comparisons:</p>
<h2>luminance vs. glium</h2>
<p><a href="https://crates.io/crates/glium">glium</a> has been around for a while – way longer than <a href="https://crates.io/crates/luminance">luminance</a>.</p>
<blockquote>
<p>According to crates.io, <a href="https://crates.io/crates/glium">glium</a> is <a href="https://users.rust-lang.org/t/glium-post-mortem/7063">no longer actively developed by its original author</a>.</p>
</blockquote>
<ul>
<li>It has more macros to generate <code>impl</code>, which is neat – however, this tends to be less and less
true as <a href="https://crates.io/crates/luminance">luminance</a> has been getting new macros lately.</li>
<li>It addresses the OpenGL concepts more directly. For instance, you find a <code>glium::VertexBuffer</code>
type – a buffer to hold vertices – while <a href="https://crates.io/crates/luminance">luminance</a> gives you a <code>Tess</code> type that hides that kind
of complexity away for you.</li>
<li><a href="https://crates.io/crates/glium">glium</a> doesn’t prevent you to pick a specific version of OpenGL / GLSL if you want to. For
instance, the <a href="https://github.com/glium/glium/blob/master/book/tuto-02-triangle.md#program">tuto-02-triangle.md</a> sample uses GLSL140. <a href="https://crates.io/crates/luminance">luminance</a> is different on that
side as it <strong>must</strong> be used with an <strong>OpenGL 3.3 Core Context</strong> – hence a GLSL330. This can be
seen as a <a href="https://crates.io/crates/luminance">luminance</a> drawback as it will only work with a specific version of OpenGL (GL 3.3 and
GLSL 330).</li>
<li>Drawing is done by requesting a <code>Frame</code> object in <a href="https://crates.io/crates/glium">glium</a> and passing the whole pipeline at once.
The pipeline then must be completely resolved prior to render. See
<a href="https://docs.rs/glium/0.20.0/glium/trait.Surface.html#tymethod.draw">this</a>. <a href="https://crates.io/crates/luminance">luminance</a> is more
flexible on that side since it enables you to provide the rendering pipeline in a dynamic way,
allowing for tricky structure traversal without cloning nor borrowing, for instance. <a href="https://crates.io/crates/glium">glium</a>
requires a tuple of <code>(vertices, indices, program, uniforms, render_state)</code> while <a href="https://crates.io/crates/luminance">luminance</a> works
by building the AST node by node, level by level.</li>
<li>The program customization (uniforms) feature of <a href="https://crates.io/crates/glium">glium</a> is per-value while <a href="https://crates.io/crates/luminance">luminance</a> is
per-type. In <a href="https://crates.io/crates/glium">glium</a>, you can change the uniforms you pass to a program by using the <code>uniform!</code>
macro when invoking the <code>draw</code> command on your <code>Frame</code>. This has the effect to lookup the uniform
in the shader for every call – modulo caching – and requires you to explicitly pass the uniforms
to use. <a href="https://crates.io/crates/luminance">luminance</a> uses a <em>contravariant</em> approach here: you give the type of the <em>uniform
interface</em> you want to use and you will be, later, handed back a reference to an object of that
type so that you can alter it. All of this is done at the type level; you have nothing to do when
passing values to the shader in the pipeline. Opposite approaches, then. Also, notice that
<a href="https://crates.io/crates/luminance">luminance</a> looks up the uniform names exactly once – at <code>Program</code> creation – and provides several
hints about uniform utilization (inactive, type mismatch, etc.). <a href="https://crates.io/crates/glium">glium</a> seems to have runtime
type checking as well, though.</li>
<li>Linear algebra seems to be handled the same way (e.g. 4×4 matrices are encoded as <code>[[f32; 4] 4]</code>
in both crates).</li>
<li>Textures are handled a bit similarily, even though, because <a href="https://crates.io/crates/glium">glium</a> uses a fixed pipeline, the
user doesn’t have to do anything special but passing references around. <a href="https://crates.io/crates/luminance">luminance</a> exposes a
scoped API to use opaque texture values, which adds more noise. A good point for <a href="https://crates.io/crates/glium">glium</a> there
since the texture passing is just about references while <a href="https://crates.io/crates/luminance">luminance</a> has a specific API for that.
This is the same situation for GPU buffers.</li>
<li>About vertices, <a href="https://crates.io/crates/glium">glium</a> is more permissive about how data can be stored on the GPU. For instance,
it supports <em>deinterleaved vertices</em> – e.g. when you have your positions in
a GPU buffer and the normals or colors in another. <a href="https://crates.io/crates/luminance">luminance</a> requires you to interleave your
vertices. This is due to the fact that if you use all your vertex attributes in a shader – which
was an assumed situation when designing the vertex API – this will provide better performances
than using deinterleaved data (because of cache friendliness). However, for cases when you don’t
want to use other attributes – depth peeling, shadow mapping, z-pre-pass, etc. – that might be a
huge drawback as you’ll peek / load data from the buffer you won’t even use, killing the point of
having a cache-friendly peek.</li>
<li>Draw state (i.e. depth test, depth buffer comparison equations, culling, etc.) is done pretty much
the same way in both crates: by creating an object representing the state when calling the
rendering pipeline.</li>
<li><a href="https://crates.io/crates/glium">glium</a> supports <em>compute shaders</em> while <a href="https://crates.io/crates/luminance">luminance</a> doesn’t (so far! ;) ).</li>
<li>Additionnally, because of the dynamic nature of its pipelines, <a href="https://crates.io/crates/luminance">luminance</a> allows for better
sharing of resources, especially shaders and render states. This is due to the fact that
<a href="https://crates.io/crates/luminance">luminance</a> represents rendering pipelines as dynamic ASTs in which you can change all the
branches and leaves on the fly. See the point about the <code>Frame</code> of <a href="https://crates.io/crates/glium">glium</a> above.</li>
</ul>
<blockquote>
<p>Sources and tutorials for <a href="https://crates.io/crates/glium">glium</a> <a href="https://github.com/glium/glium/tree/master/book">here</a>.</p>
</blockquote>
<h2>luminance vs. gfx</h2>
<p><a href="https://crates.io/crates/gfx">gfx</a> is a high-performance, bindless graphics API for the Rust programming language. It aims to be
the default API for Rust graphics: for one-off applications, or higher level libraries or engines.</p>
<blockquote>
<p>The current listing concerns <a href="https://crates.io/crates/gfx">gfx</a> in its’ <em>pre- low level</em> situation. Recent <a href="https://crates.io/crates/gfx">gfx</a> APIs are
different.</p>
</blockquote>
<ul>
<li>The <a href="https://crates.io/crates/gfx">gfx</a>’s scope is not the same as <a href="https://crates.io/crates/luminance">luminance</a>’s: <a href="https://crates.io/crates/gfx">gfx</a> abstracts over <strong>several graphics API</strong>.
Among them, <strong>OpenGL</strong>, <strong>Metal</strong>, <strong>DirectX</strong> and will clearly attract people who want to <em>write
once, release everywhere</em>.</li>
<li><a href="https://crates.io/crates/gfx">gfx</a> requires you to write shader sources for all possible backends. See for instance this
<a href="https://github.com/gfx-rs/gfx/tree/pre-ll/examples/cube/shader">example</a> which uses
GLSL, HLSL, Metal and even SPIR-V – it even uses several version of GLSL, running in GLSL 100, 120
and 150.</li>
<li>Vertex types are defined differently from <a href="https://crates.io/crates/luminance">luminance</a>. In <a href="https://crates.io/crates/gfx">gfx</a>, you use the <code>gfx_defines!</code> macro
to easily introduce the types and make named bindings (to map to vertex attributes in shaders). A
lot of boilerplate is generated thanks to that macro, which is neat. <a href="https://crates.io/crates/luminance">luminance</a> has a similar
scheme even though the macro support in <a href="https://crates.io/crates/luminance">luminance</a> is less advanced. On a general note:
<a href="https://crates.io/crates/luminance">luminance</a> is less macro-driven.</li>
<li>Similarily, the uniforms are handled in the same <code>gfx_defines!</code> macro. They’re declared in very
similar ways as in <a href="https://crates.io/crates/luminance">luminance</a>. However, <a href="https://crates.io/crates/gfx">gfx</a> uses the same syntax as with vertex definition,
while <a href="https://crates.io/crates/luminance">luminance</a> has an ad hoc approach – it’s typical Rust code with <em>annotations</em>. That doesn’t
make a huge difference per-se. <a href="https://crates.io/crates/gfx">gfx</a> calls <em>uniforms</em> <code>constant</code>, which is nice because it can
abstract over the concept, while <a href="https://crates.io/crates/luminance">luminance</a> doesn’t and uses the terms <code>Uniform</code>, <code>Uniformable</code>,
<code>UniformInterface</code>, etc.</li>
<li>The pipeline system in <a href="https://crates.io/crates/gfx">gfx</a> is also defined in <code>gfx_defines!</code> macro invokations. This is
seriously cool and <a href="https://crates.io/crates/luminance">luminance</a> suffers a bit from not having that – defining pipeline in
<a href="https://crates.io/crates/luminance">luminance</a> is a bit boring because of lambda / closures boilerplate.</li>
<li>In <a href="https://crates.io/crates/gfx">gfx</a>, the GPU buffers are handled via <em>factories</em> and smart constructors, like
<code>create_vertex_buffer_with_slice</code>, for instance. This is nice as it enables coping with
multi-threading safety, which is something <a href="https://crates.io/crates/luminance">luminance</a> doesn’t support yet (it’s planned, though).</li>
<li><a href="https://crates.io/crates/gfx">gfx</a> comes in with an <em>application</em> abstraction layer, which <a href="https://crates.io/crates/luminance">luminance</a> doesn’t – 
<code>gfx_app::Application</code>, defined in the <a href="https://crates.io/crates/gfx_app">gfx_app</a> crate.</li>
<li><a href="https://crates.io/crates/gfx">gfx</a> has <strong>a lot</strong> of running projects and is actively maintained and changed over time. For
instance, both <a href="https://crates.io/crates/ggez">ggez</a> and <a href="https://crates.io/crates/amethyst">amethyst</a> use <a href="https://crates.io/crates/gfx">gfx</a>.</li>
</ul>
<blockquote>
<p>On a general note, <a href="https://crates.io/crates/luminance">luminance</a> and <a href="https://crates.io/crates/gfx">gfx</a> pre-ll are way too different to be compared to each
other. If you want to target several backends, do not hesitate and go to <a href="https://crates.io/crates/gfx">gfx</a>. If you target only
one system and want simplicity, go to <a href="https://crates.io/crates/luminance">luminance</a>.</p>
</blockquote>
<h2>luminance vs. ggez</h2>
<p><a href="https://crates.io/crates/ggez">ggez</a> is a Rust library to create a Good Game Easily.</p>
<blockquote>
<p>As <a href="https://crates.io/crates/ggez">ggez</a> is primarily focused on 2D projects, it doesn’t compare directly to <a href="https://crates.io/crates/luminance">luminance</a>, as it’s
about graphics, sound, input, etc. Also, <a href="https://crates.io/crates/ggez">ggez</a> is using <a href="https://crates.io/crates/gfx">gfx</a>.</p>
</blockquote>
<h2>luminance vs. vulkano</h2>
<p><a href="https://crates.io/crates/vulkano">vulkano</a> is a wrapper around the Vulkan graphics API.</p>
<blockquote>
<p>The scope is not really the same as <a href="https://crates.io/crates/vulkano">vulkano</a> only targets Vulkan and – as for today – <a href="https://crates.io/crates/luminance">luminance</a>
is about OpenGL.</p>
</blockquote>
<h2>luminance vs. amethyst</h2>
<p><a href="https://crates.io/crates/amethyst">amethyst</a> is a fast, data-oriented, and data-driven game engine suitable for rapid prototyping and
iteration.</p>
<blockquote>
<p>As stated in the current document, <a href="https://crates.io/crates/luminance">luminance</a> is not a game engine. So if you’re looking for a
tool that already has everything needed to build a game, <a href="https://crates.io/crates/amethyst">amethyst</a> should be your toy. Also, it’s
using <a href="https://crates.io/crates/gfx">gfx</a>.</p>
</blockquote>
<h2>luminance vs. piston</h2>
<p><a href="https://crates.io/crates/piston">piston</a> is a modular game engine written in Rust.</p>
<blockquote>
<p>As for other game engines, nothing to compare here since <a href="https://crates.io/crates/luminance">luminance</a> is <strong>not a game engine</strong>!</p>
</blockquote>
<h2>luminance vs. spectra</h2>
<p><a href="https://crates.io/crates/spectra">spectra</a> is a demoscene framework / engine mainly written for demoscene purposes, even though it
can now be used to some other extents (animation / video game).</p>
<blockquote>
<p><a href="https://crates.io/crates/spectra">spectra</a> uses <a href="https://crates.io/crates/luminance">luminance</a> as primary <em>graphics backend</em>, but it provides way more features than
just graphics. You’ll find audio, animation, editing, resource system, custom shading language,
etc.</p>
</blockquote>


guid: https://strongly-typed-thoughts.net/blog/luminance-0.40Thu, 16 Jul 2020 18:49:00 GMT
title: The new luminance is there
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance-0.40
feedurl: https://phaazon.net/blog/feed
content:
<p>Today is the day <a href="https://crates.io/crates/luminance/0.40.0">luminance-0.40</a> got released. This is a <em>massive</em> update. Like never before in
the luminance ecosystem.</p>
<h1>(Short) story of what happened</h1>
<p>A huge amount of features, bug fixes, usability / comfort enhancement, new crates and architecture
redesign are gathered in that update. First, let’s talk about the new most important feature: the
redesign.</p>
<h2>The backend (re)design</h2>
<p>Before 0.40, luminance was an <a href="https://www.opengl.org">OpenGL 3.3</a> crate. If you’re not into graphics APIs, OpenGL is a
graphics normalized specification, built by an open group (Khronos) that has been around for
decades. Because it’s an open  specification, lots of vendors can implement and provide drivers to
have OpenGL support, but also Free and OpenSource projects. The 3.3 version was old enough to
support a wide variety of devices, even old ones, while still having interesting features allowing
to do lots of useful things.</p>
<p>However, as time passes, new devices and technologies emerge. <a href="https://www.khronos.org/registry/webgl/specs/latest/1.0">WebGL 1.0</a>, which appeared in 2011,
is to the Web what OpenGL is to desktop and console graphics programming. It is an open API
browsers can implement to provide support for accelerated 2D/3D rendering in your browser.</p>
<p><a href="https://www.khronos.org/registry/webgl/specs/latest/2.0">WebGL 2.0</a>, which appeared more recently, in 2017 (partial support in some browsers), provide an
API almost identical to what you find in OpenGL 3.3 (with some tricky caveats).</p>
<p>I’ve been wanting to have a way to write “luminance code”, and have it compile for other
platforms than desktops, like Web and mobile (Android and iOS). With the previous architecture,
since luminance was bound to OpenGL 3.3, it was not possible. Here comes the new archicture.</p>
<h2>Backends… backends? backends!</h2>
<p>The whole luminance crate was rewritten so that its public-facing interface is completely agnostic
of what <em>executing GPU code</em> is about. No OpenGL anymore. Instead, it depends on a type variable,
referred as <code>B</code> in luminance’s types, which must satisfy a set of conditions to be considered a
<em>backend</em>.</p>
<p>A backend is an implementation of the expectations made on the public-facing interface. For
instance, you can <code>set</code> a GPU buffer by providing an index and a value. That’s the public facing
part. However, what happens on the GPU is left to the backend implementation.</p>
<p>This new way of doing split the luminance crate into several ones:</p>
<ul>
<li><a href="https://crates.io/crates/luminance">luminance</a>, which is the core, more abstract and common crate to whole. You use types coming
from this crate to explain <em>what you want to do</em>. How it’s done is made by other crates.</li>
<li><a href="https://crates.io/crates/luminance-gl">luminance-gl</a>, which is the OpenGL backend crate. It roughly contains a type — <code>GL33</code> — which
can be passed around <a href="https://crates.io/crates/luminance">luminance</a> to select the OpenGL 3.3 backend.</li>
<li><a href="https://crates.io/crates/luminance-webgl">luminance-webgl</a>, which is the WebGL 2.0 crate. I decided to go with WebGL 2.0 instead of 1.0
because of how similar it is to OpenGL 3.3 (which was the existing code base). It’s possible to
add support for WebGL 1.0, but it will require a lot of effort to support the feature that are
not present in the specification.</li>
</ul>
<p>Added to this, because of how generic and abstract <a href="https://crates.io/crates/luminance">luminance</a> now is, people might get issues when
trying to write code that will work whatever the backend. When using the <code>set</code> functions on a
buffer, it is required that the type implements a trait from <a href="https://crates.io/crates/luminance">luminance</a> —
<code>luminance::backend::buffer::Buffer</code>. So it’s likely the user will have to constrain types and it
might get boring. For this reason, a new crate was created: <a href="https://crates.io/crates/luminance-front">luminance-front</a>.</p>
<p><a href="https://crates.io/crates/luminance-front">luminance-front</a> allows people to write code using types from <a href="https://crates.io/crates/luminance">luminance</a> that got their <code>B</code> type
variable replaced by a backend type at compile-time. This is done by inspecting the compilation
target, and features you enable. You are then free to write your code without worrying about
whether traits are implemented, since <a href="https://crates.io/crates/luminance-front">luminance-front</a> takes care of that for you. The
re-exported types are simple aliases to <a href="https://crates.io/crates/luminance">luminance</a>’s types, so your generic code — if you write
any — will be compatible.</p>
<h2>Changelog, migration guide</h2>
<p>The changelog is quite massive for a single update. I wish I had the opportunity to split it into
several updates, but the redesign meant a lot of changes, and I got several very interesting PRs
and feature requests. Among very new stuff:</p>
<ul>
<li>A new platform crate has appeared: <a href="https://crates.io/crates/luminance-sdl2">luminance-sdl2</a>, which adds support for the <a href="https://crates.io/crates/sdl2">sdl2</a> crate.</li>
<li><a href="https://crates.io/crates/luminance-webgl">luminance-webgl</a> and <a href="https://crates.io/crates/luminance-web-sys">luminance-web-sys</a>, to support the Web!</li>
<li><a href="https://crates.io/crates/luminance-front">luminance-front</a>, which is a <em>front</em> crate to ease working with <a href="https://crates.io/crates/luminance">luminance</a> types.</li>
<li>The type system experience has been greatly improved. Most of the time, you will not have to
annotate types anymore — like <code>Program</code> or <code>Tess</code>.</li>
<li>About <code>Tess</code>, a BIG update has landed, has it’s now heavily typed (vertex type, index type,
vertex instance data type, memory interleaving type).</li>
<li>More render states features, such as the possibility to enable or disable depth writes, separate
RGB/alpha blending, etc. etc.</li>
</ul>
<p>A huge thanks to all the people who contributed. It means a lot to me! The list of changes is big,
so I’ll leave you with the <a href="https://github.com/phaazon/luminance-rs/blob/master/luminance/CHANGELOG.md#040">changelog here</a>.</p>
<blockquote>
<p>Disclaimer: this is the <a href="https://crates.io/crates/luminance">luminance</a> changelog, but all the crates have their own, too.</p>
</blockquote>
<p>Because that update is massive and has some breaking changes, I have also decided to include a
<em>migration guide</em>. The migration guide is a section in <a href="https://crates.io/crates/luminance">luminance</a>’s changelog to explain all
the things you need to do to migrate from luminance-0.39 to luminance-0.40. <strong>Please: if you find
something that is missing or you’re struggling with, please open an issue, a PR, or ping me on
whatever platform you like</strong>.</p>
<h2>What’s next to read</h2>
<p>The <code>Tess</code> type now supports slicing deinterleaved memory in very, very elegant ways, checked
at compile-type and type-driven. The whole new <code>Tess</code> type design is so new that a blog article
is on its way about that topic, as I find it very interesting.</p>
<p>While testing the redesign and new features, I have implemented a <a href="https://en.wikipedia.org/wiki/Conway%27s_Game_of_Life">Conway’s Game of Life</a> for
fun — no code available yet. I plan to remake it from scratch and record the whole experience
on video. That will be a new format and I want to hear from people and know whether they like the
format.</p>
<p>Finally, the <a href="https://rust-tutorials.github.io/learn-luminance">book</a> was also updated. The three chapters have been updated and some features have
been added — like aspect ratio in chapter 3; I don’t understand why I haven’t talked about that
earlier!</p>
<p>The next steps for luminance and myself, besides the blog articles and video, are… getting some
rest. I plan to get back to demoscene production, and I’m 100% sure there’s already a lot to do
with that current release of <a href="https://crates.io/crates/luminance">luminance</a>. I plan to experiment with new backends as well, such as
an OpenGL 2.0 backend (if possible), to support really old hardware; an OpenGL 4.6 backend for
modern hardware (easy), as I already know that API pretty well; an OpenGL ES backend for mobile
development, and later, a <a href="https://www.khronos.org/vulkan">Vulkan</a> and, perhaps, a <a href="https://gpuweb.github.io/gpuweb">WebGPU</a> backends.</p>
<p>Stay tuned for the <code>Tess</code> article, the Game of Life video… and keep making super fun video games,
animation and toys with <a href="https://crates.io/crates/luminance">luminance</a>!</p>
<p><em>And keep the vibes!</em></p>


guid: https://strongly-typed-thoughts.net/blog/aerc-2024Wed, 05 Jun 2024 10:12:00 GMT
title: Moving away from PR/MR workflow
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/aerc-2024
feedurl: https://phaazon.net/blog/feed
content:
<h1>Moving away from PR/MR workflow…</h1>
<p>In the past two years, many projects of mine got more and more contributions.
Some of those projects, such as <strong>This Week in Neovim</strong> (that I gave away),
<code>kak-tree-sitter</code> and others received many contributions. As I reviewed
contributions, I realized that I do not really enjoy the tools used to implement
the process. That is, GitHub. Moreover, I have a special thing against
Microsoft (in)famous way of doing things in (c.f. <a href="https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish">EEE</a>). So, why is it such a
big deal to me, and what is the alternative?</p>
<h2>What’s wrong with git host platforms (GitHub, GitLab, etc.)</h2>
<p><code>git</code> (I don’t think I need to explain what it is, right?) was created by Linus
Torvalds (again, I don’t think I need to present him) to solve the <a href="https://en.wikipedia.org/wiki/BitKeeper#BitKeeper_and_the_Linux_Kernel">BitKeeper
dispute</a>. Back then, people were using CVS or SVN, and something like BitKeeper
was a centralized place where the kernel was developped.</p>
<p>In 2005, after the decision to stop providing free BitKeeper copies to the
kernel community, Linus created <code>git</code>. He created <code>git</code> to provide the free and
open-source software community with a versioning tool working on a
decentralized model. Indeed, with <code>git</code>, teams can organize the way they want
and exchange patches and commits without having to depend on a third party.</p>
<p>For instance, if Alice and Bob want to work on a project together and version
it via <code>git</code>, they do not need to rely on <em>anyone else</em> but themselves. They can
send commits to each other with <code>git push</code> or <code>git pull</code> (granting they
configure their <code>ssh</code> correctly). They can use their university server to host
a bare git repository and synchronize there. They can exchange patches by mail.
Etc.</p>
<p>Now, something else they could do would be to use GitHub. Or GitLab. Or
BitBucket. In the end, it’s the option mentioned above: a repository always up
for people to send commits to and receive commits from. However, I do see one
<em>weird</em> pattern here. Using GitHub, for instance (but really, it’s the same for
most others) is basically the same as taking a decentralized tool and putting it
in jail. If someone wants to contribute to Alice and Bob project, they have to
create a GitHub account, and abide by the GitHub rules. More interesting, GitHub
is not a free and open-source project anymore. It’s owned by Microsoft. Alice
and Bob projects are physically stored on a Microsoft server. Given the history
of Microsoft in the FOSS world, I can imagine how it would cause a moral
problem.</p>
<p>Another point that is specific to GitHub: Copilot. I think Copilot is FOSS
abomination. The reason is that, even though your code is under a copyleft — or
even a copyright, <a href="https://docs.github.com/en/site-policy/github-terms/github-terms-of-service">GitHub terms of service</a> is pretty explicit about the hosted
content: they can use it to improve their own products.
<a href="https://docs.github.com/en/site-policy/github-terms/github-terms-of-service#d-user-generated-content">This section</a>
for instance. I don’t know about you, but to me, it’s grey zone there. I do not
like the idea that my code can be “analyzed” by Microsoft to enhance Copilot.
All of my projects are licensed under a copyleft (BSD-3) that mentions that my
name should be mentioned when my code is used, and other clauses that I think
are violated by this Copilot horror.</p>
<p>And I’m not even talking about <a href="https://www.microsoft.com/en-us/store/b/copilotpro">Microsoft Copilot Pro</a>, which is a product. That
you have to pay for. That is made from public, free and open-source
contributions from millions of projects. <em>Fuck that</em>.</p>
<p>Finally, those platforms often host many different things that people will start
using and highly depend on. For instance, bug tracker, wikis, project
management, etc. Even though this is not the main problem to me, I like using
something that was made to solve a single problem. Having a giantic platform
solving basically “How to do software development” is something that now sounds
like a red flag to me. What happens when you decide to move on? Well, you have
to move <em>everything</em>. And it’s my case today.</p>
<h2>Going back to a decentralized way</h2>
<p>Something like two years ago, someone on IRC praised the email-based git
workflow. Wait, <em>email</em>-based? In 2022? Isn’t that a bit too hardcore? Well,
there is one really big misconception about emails: all-in-one providers.</p>
<p>Like many other people, I do use a webmail. If it’s not GMail, it’s Outlook or
something else. GMail is hard not to notice, because in the professional world,
every companies use it. It’s easy for them to manage and you get all the
important things at once — swarm of newcomer welcoming emails, meetings, various
meetups, etc. All in one. The problem with that is that it requires you to go
to your browser. And the online experience is honestly pretty bad — it always
has been to me. I try to spend as few times as possible in GMail, because I
don’t really like the interface.</p>
<p>So how can someone use emails to work with <code>git</code>? Well, from the beginning,
<code>git</code> was made with the email workflow on mind. The idea is simple:</p>
<ol>
<li>You make your changes exactly the same way as you normally do.</li>
<li>When you have your commits and you are ready to share your contributions with
others for review, instead of going to a centralized platform to push your
branch to and then open a PR, you simply use the <code>git send-email</code> command —
or <code>git format-patch</code> and manually send the patch, but really, you shouldn’t
have to do that. You typically send that email to a development mailing list.</li>
<li>People discuss the patch by replying to the emails.</li>
<li>Either you send other patches via other mails, or a maintainer eventually
approves and <em>applies</em> (forget about <em>merging</em> here!) the patches with
<code>git am</code>.</li>
<li>Done.</li>
</ol>
<h3>Misconception 1: reviewing is hard</h3>
<p>The main misconception part is about the <em>review</em> part. On GitHub, reviewing is
nice, because you can comment in line. In emails… well, it’s exactly the same,
and even better. What people usually do is to reply to your email by quoting it,
and dissect the email section by section. For instance, imagine I have a project
with a <code>README.md</code> file containing a single line:</p>
<pre><code class="language-markdown">Hello. My name is Dimitri and this is a README.
</code></pre>
<p>Let’s say I modify that file to hold this content instead:</p>
<pre><code class="language-markdown"># Hello world!

My name is Dimitri and this is a README.
</code></pre>
<p>I make a commit with <code>git commit -am &quot;Add hello section to README.md.&quot;</code>. When
using <code>git send-email</code> / <code>git format-patch</code>, it will generate something like
this patch:</p>
<pre><code>From 020d9085a35d6c34bc1b7e86b78f257c4d556bbb Mon Sep 17 00:00:00 2001
From: Dimitri Sabadie &lt;hadronized@strongly-typed-thoughts.net&gt;
Date: Thu, 30 May 2024 14:28:11 +0200
Subject: [PATCH] Add hello section to README.md.

It looks better to me and allows to demonstrate the email workflow a bit more.
---
 README.md | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 72ae80f..70341da 100644
--- a/README.md
+++ b/README.md
@@ -1 +1,3 @@
-Hello. My name is Dimitri and this is a README.
+# Hello world!
+
+My name is Dimitri and this is a README.
--
2.45.1
</code></pre>
<p>The first 4 lines are part of the email, so when receiving your email, the
client of the people reading the mailing list should format all that correctly.
The rest is the email content! Someone could reply something like:</p>
<pre><code>&gt; -Hello. My name is Dimitri and this is a README.
&gt; +# Hello world!

Thank you, we needed that header.

&gt; +My name is Dimitri and this is a README.

May you add more info to include other maintainers as well? Alice and Bob could
provide more details there.

Cheers,
Dimitri
</code></pre>
<p>The author of the patch can then iterate and send another email (with the <code>-v2</code>
annotation, because it would be the next version; <code>-v3</code> for the next iteration,
etc.). The advantage here is that someone can reply to that first review by just
discussing the first part, quoting it:</p>
<pre><code>&gt; &gt; -Hello. My name is Dimitri and this is a README.
&gt; &gt; +# Hello world!
&gt;
&gt; Thank you, we needed that header.

I think we should also include a sub-header describing something else.
</code></pre>
<p>While at the same time, someone can discuss the second part of the change:</p>
<pre><code>&gt; &gt; +My name is Dimitri and this is a README.
&gt;
&gt; May you add more info to include other maintainers as well? Alice and Bob could
&gt; provide more details there.

Yes, like email addresses and PGP signatures, please.
</code></pre>
<p>Those will create a <em>tree</em> of emails, where each sub-thread is scoped to a
specific part of your patch. Such a feature is not possible in GitHub or any
other centalized platform.</p>
<p>So the only important thing is to use a good client. I personally use <a href="https://aerc-mail.org/">aerc</a>,
along with some other tools to synchronize IMAP maildirs and send SMTP messages.
The configuration is not that hard once you get the hang of it.</p>
<h3>Accounts?</h3>
<p>Sending an email to contribute is nice, but where to send the email? Well, it
depends on the project. For that, I think that every open-source projects should
have a <code>CONTRIBUTING.md</code> file explaining where to send patches. I use
<a href="https://sourcehut.org/">SourceHut</a>, which provides me a place to host my repository and mailing lists.
The rest is done entirely outside of SourceHut. Especially, even though you can
create an account on SourceHut, <strong>you don’t have to</strong>. You don’t have to in
order to:</p>
<ul>
<li>Contribute code. You can just send the patches to the mailing list. <strong>You
don’t need an account!</strong>.</li>
<li>Open a bug report or a feature request. You can just send your bug report or
feature request to… the bug report mailing list or the feature request mailing
list! <strong>No account needed!</strong>.</li>
<li>Etc. etc.</li>
</ul>
<p>That part here makes me smile, because it makes so much more sense to me. People
can contribute with their email address. They do not need to engage with the
platform to start contributing, which is nice and more <em>decentralized</em> to me.
One can even send a patch to a maintainer directly if they really want.</p>
<h3>Authorship</h3>
<p>I think the problem with the email-based workflow is that you don’t exchange
<em>commits</em> anymore, but <em>patches</em>. There is a slight difference.</p>
<p>A <em>patch</em> is a set of changes to be applied. It contains some metadata, like
the parent commit to apply to, the expected SHA1, some diff-stats, and that’s
pretty much all.</p>
<p>A <em>commit</em> is a patch with more metadata, like the committer / author
identities, a commit message, the parent commit, a signature, etc. etc.</p>
<p>A commit always refers to at least one parent (unless it was the first commit).
When you send a patch over email, the maintainer that applies the patch does it
with <code>git am</code>, which turns the email content into a commit. The message commit,
for instance, is taken from the mail body. However, there is one information
<strong>that is lost during the process</strong>: the signature.</p>
<p>Indeed, when you work on your local copy of the repository, you might sign your
commits with your PGP key. When you decide to send the commits as patches to a
development list, all of the metadata from the commits are ripped. That means
that the signature is simply lost.</p>
<p>For most people, in terms of security, it’s not that bad. Patches are accepted
from emails, and you can (you should!) sign your email with your PGP key. As a
maintainer, my email client (<code>aerc</code>) shows me the signature. If you don’t sign
your email, I cannot certify it’s you sending me the email. Most people don’t
seem to care that much about authorship, but I do. I think it’s important to be
able to retrace that a patch which identity is <code>foo@bar.net</code> was really provided
by that person. A signature doesn’t prove you wrote the patch, but it proves
you submitted it, the same way that signing a commit means that you integrated
that commit into the repository.</p>
<p>Currently, there is no way to do that on the email workflow and I find that a
bit of a problem. There should be an alternative, like manually signing the
patch. The problem is that, if the maintainer has to use a three-way merge, it
will invalidate both the parent commit’s SHA1 and the expected SHA1 of the
patch, invalidating the carried signature.</p>
<h3>In the end</h3>
<p>Even though the email workflow has some problems, having the contributions
locally and applying patches locally is a much more enjoyable process. The first
thing is that sending many commits ends up in a nice thread of mails. It’s super
easy to just open the thread locally and apply the patches. The only concern I
have is authorship, but people told me I might be the only person caring about
that — am? All in all, this email-based workflow allows me to do everything
from my terminal, and even <em>tickets</em> (todos in sourcehut) and bug reports /
feature requests are done via email. It just feels good.</p>


guid: https://strongly-typed-thoughts.net/blog/rust-features-documentationSat, 13 Oct 2018 23:37:00 GMT
title: Rust and features discoverability
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/rust-features-documentation
feedurl: https://phaazon.net/blog/feed
content:
<p>I’ve been working on several projects lately and writing a bit about them on my blog – especially,
specific technical problems. This very blog entry is on a different level though: it’s about our
tooling in Rust.</p>
<p>See, most tools in Rust are wonderful:</p>
<ul>
<li><code>rustc</code>, the official compiler, is pretty easy to use and even though it’s an advanced piece of
software, it’s pretty unlikely you’ll have to use it directly (unless you’re doing something
very exotic, or working on it directly).</li>
<li><code>cargo</code>, by far the most appealing tool in the Rust ecosystem to me, is a Rust build system and
package manager. It downloads and checks your projects’ dependencies, using https://crates.io as
a default mirror for your crates (but you can
<a href="https://doc.rust-lang.org/cargo/reference/unstable.html#alternate-registries">use other mirrors if you want and are brave enough</a>,
even yours), (cross) compiles, runs library, binary and even documentation tests, etc.</li>
<li><code>cargo</code> also comes with a plugin system, allowing people to write cargo <em>subcommands</em> that can
be installed like with <code>cargo install cargo-tree</code> and run with <code>cargo tree</code>. Most common plugins
are:
<ul>
<li><code>bloat</code>.</li>
<li><code>watch</code>.</li>
<li><code>tree</code>.</li>
<li><code>clippy</code>.</li>
<li><code>fuzz</code>.</li>
<li><code>expand</code>.</li>
<li><code>outdated</code>.</li>
<li>List <a href="https://github.com/rust-lang/cargo/wiki/Third-party-cargo-subcommands">here</a>.</li>
</ul>
</li>
<li><code>rustup</code>, used to select, switch, install and update Rust compilers, toolchains, support
cross-compilation and select targets, etc.</li>
<li><a href="https://github.com/rust-lang-nursery/rls">RLS</a>.</li>
<li>Plus some additional, non-local tools such as
<ul>
<li><a href="https://play.rust-lang.org">Rust Playground</a>.</li>
<li><a href="https://crates.io">crates.io</a>, obviously.</li>
<li>The <a href="https://doc.rust-lang.org/std">excellent Rust documentation</a>, giving you information and
documentation about the standard library, the procedural macro, the allocation system, etc.</li>
<li>The <a href="https://docs.rs">docs.rs</a> documentation host, hosting ALL crates’ documentation for free,
automatically and with fuzzy searching.</li>
</ul>
</li>
<li>And many others.</li>
</ul>
<p>Clearly, you can see we have lots of wonderful tools made by talented people and you can clearly
feel how easy it is to start writing a crate, test it and publish it along with its documentation.
It feels seamless. The documentation is a very important thing in any tech ecosystem for several
reasons. And this blog is going to be about doc tooling and some specific parts of Rust.</p>
<h1>Documentation is key to achieve good quality… but not only</h1>
<p>Whatever the project you work on, you ~should~ <strong>must</strong> document your code. There are several
situations – let’s call this the <strong>First Hypothesis</strong>:</p>
<p>If you write code that is intended to be released publicly, you must document every public
symbols:</p>
<ul>
<li>Structs, enums and type aliases.</li>
<li>Any public field.</li>
<li>Constants.</li>
<li>Functions and macros.</li>
<li>Traits along with their associated types / constants and methods.</li>
<li>Trait impls (think of how useful the documentation of an <code>impl Default</code> can be!).</li>
<li>Modules.</li>
</ul>
<p>Now, if you are writing something that is private to a crate that you plan to release, think about
people who will want to contribute to help you with the crate. They will have to get to read your
code. They will have to go through the (sometimes painful) process of adapting to someone else’s
way to write code. So do them a favor: document your code.</p>
<p>Finally, what about things you don’t plan to release? Like, for instance, a binary? The same rule
actually applies: what happens if someone tries to contribute? What if you save the project aside
for some weeks and get back to it afterwards?</p>
<p>Those three paragraphs just show you that the <strong>First Hypothesis</strong> was wrong: there are not several
situations. Only one: as soon as you write code, whatever whom it’s aimed at, just document it.
You’ll be thanked and you might even thank yourself in a near future for doing so. Documenting your
code enables you to run <code>cargo doc --open</code> and immediately start <em>exploring</em>. Exploring is when you
need to catch up with a project’s ideas, concepts, or whenever you join a team as a new job and need
to get used to the codebase.</p>
<p>I want to share my experience here: most people are <em>bad</em> at this – it’s not necessarily their
faults, don’t blame them. Most teams I worked in were under high pressure, with business deadlines
to meet – I’ve been there and still am. This is more about a political discussion to have with “the
ones who give you such deadlines” but really, <strong>developing a project doesn’t stop at writing code
and testing.</strong> Onboarding and discoverability should be considered of a massive importance, because
it helps preventing the project from burying itself and getting too bloated for newcomers or even
long-running team members to understand <em>the actual heck is happening there</em>. When someone new joins
your team, you should help them to get accustomed to the codebase… but you also have to work. They
should have <em>leads</em> or <em>hints</em> to get their feet wet. Several solutions exist to document that in
<em>kind of</em> standardized ways:</p>
<ul>
<li>Write a <code>README.md</code> in all the projects your team has. This should be mandatory. Describe what
the project is about, how to build it as a developer, how to run it as a dev-ops. Describe the
architecture, etc.</li>
<li>Write a <code>CONTRIBUTING.md</code> to help onboarding newcomers with your team guidelines, conventions,
engineering processes, etc.</li>
<li>Be consistent: it’s not because a small project consists in only a single shell script that you
shouldn’t write a <code>README.md</code>: <strong>write it</strong>!</li>
<li>Something that I love doing, especially at work: have some kind of a cron / task that builds the
documentation for all your projects across the scope of your team… and have it host it all
on a server documented in the top-level <code>README.md</code> of your team. This is a real plus: people
will just go to that documentation link to search for things <em>“that have already been done by
the team”</em> instead of reinventing the wheels – trust me, even the ones who wrote the features
might have forgotten writing them already.</li>
</ul>
<p>We’re now reaching the point of this blog entry: documentation can (should?) be used to explore what
a project is about, how to use it, what are the public variants and invariants, etc. But there is
however a problem in the current Rust ecosystem, preventing us from completely have a comfortable
exploration of a crate or set of crates. A very, very important kind of public symbol is missing
from the list of documented Rust symbols. Do you think you can figure out which one?</p>
<h1>The missing piece to exploring a crate</h1>
<p>Have you ever worked with a customized crate? If you don’t know what I’m referring to, I’m thinking
about crates that can be configured with <em>features</em>. For instance, as a good example of this, I’ll
talk about a crate of mine: <a href="https://crates.io/crates/splines">splines</a>. The crate has several features you can set to achieve
different effects:</p>
<ul>
<li><code>&quot;serialization&quot;</code>: this feature will turn on the <a href="https://crates.io/crates/serde">serde</a> serialization code (both <code>Serialize</code>
and <code>Deserialize</code>) for all the public types exposed by <a href="https://crates.io/crates/splines">splines</a>.</li>
<li><code>&quot;impl-cgmath&quot;</code>: turn this on to have some <a href="https://crates.io/crates/cgmath">cgmath</a>’s types be usable directly within <a href="https://crates.io/crates/splines">splines</a>
by implementing the appropriate traits from <a href="https://crates.io/crates/splines">splines</a>.</li>
<li><code>&quot;impl-nalgebra&quot;</code>: same thing as <code>&quot;impl-cgmath&quot;</code> but for <a href="https://crates.io/crates/nalgebra">nalgebra</a>.</li>
<li><code>&quot;std&quot;</code>: automatically enabled, you can disable it with <code>default-features = false</code> or
<code>--no-default-features</code>. When disabled, the code behaves compiled with <code>no_std</code>.</li>
</ul>
<p>All of this is currently documented in the <code>README.md</code> of the project and the top-level
documentation of the project (for instance, <code>splines-0.2.3</code> has <a href="https://docs.rs/splines/0.2.3/splines/#features-and-customization">this paragraph about features and
what they do</a>).</p>
<p>We have two problems here:</p>
<ul>
<li>It’s hard for a newcomer to <em>discover</em> and <em>explore</em> features of <a href="https://crates.io/crates/splines">splines</a> (or any crate using
features) because they’re not documented like a Rust symbol is (a function, a type, etc.), while
they truly <strong>are</strong> recognized by <code>rustc</code> and <code>cargo</code>!</li>
<li>We can <em>“bypass”</em> this problem by documenting and having a list of features like I do with
<a href="https://crates.io/crates/splines">splines</a>, but that requires the project contributors to ensure to update the list whenever they
add, remove or alter the features set.</li>
</ul>
<p>You can see that the situation is not really comfortable. I looked into the list of PRs and issues
about that topic on <a href="https://github.com/rust-lang/rfcs">rust-lang/rfcs</a> and didn’t find anything.
So I might start to write some ideas here and maybe, depending on the feedback, write a new RFC to
fix that problem.</p>
<h1>Pre-RFC: documenting and exploring crate features</h1>
<p>In order to know what modification we can do without introducing breaking changes, we need to have
a look at how features are defined. This is done in a <code>Cargo.toml</code> manifest, such as (from
<a href="https://crates.io/crates/splines">splines</a>):</p>
<pre><code>[features]
default = [&quot;std&quot;, &quot;impl-cgmath&quot;]
serialization = [&quot;serde&quot;, &quot;serde_derive&quot;]
std = []
impl-cgmath = [&quot;cgmath&quot;]
impl-nalgebra = [&quot;nalgebra&quot;]
</code></pre>
<p>A feature is a key-value object where the key is the name of the feature and the value is a list of
dependencies (crate names, crate cluster, a feature of another crate, etc. – more about that
<a href="https://doc.rust-lang.org/cargo/reference/manifest.html#the-features-section">here</a>).</p>
<p>We could document the features here. After all, that’s what Haskell does with <code>cabal</code> and <em>flags</em>:</p>
<pre><code>Flag Debug
  Description: Enable debug support
  Default:     False
</code></pre>
<p>So why not simply do the exact same in Rust within a <code>Cargo.toml</code> manifest? Like so:</p>
<pre><code>[features]
default = [&quot;std&quot;, &quot;impl-cgmath&quot;]

[features.serialization]
description = &quot;Set to automatically derive Serialize and Deserialize from serde.&quot;
dependencies = [&quot;serde&quot;, &quot;serde_derive&quot;]

[features.std]
description = &quot;Compile with the standard library. Disable to compile with no_std.&quot;
dependencies = []

[features.impl-cgmath]
description = &quot;Implement splines’ traits for some cgmath public types.&quot;
dependencies = [&quot;cgmath&quot;]

[features.impl-nalgebra]
description = &quot;Implement splines’ traits for some nalgebra public types.&quot;
dependencies = [&quot;nalgebra&quot;]
</code></pre>
<p>In order to introduce this feature in a retro-compatible way, using the <em>“old”</em> syntax would just
behave the same way but without a description, making them optional.</p>
<p>The whole purpose of this addition would be that the rustdoc team could assume the existence of an
optional <code>description</code> field on features and present them when browsing a crate’s documentation,
enriching the exploration and discovering experiences of developers. Instead of having to look at
both <code>lib.rs</code> and <code>Cargo.toml</code> to look for features, one could just simply go on https://docs.rs,
look for the crate they want to get features information <em>et voila</em>!</p>
<h2>Features consequences</h2>
<p>Some people might think of the <em>consequences</em> of features. Those are, after all, used for
conditional compilation. What that means is that part of the code might be hidden from the
documentation if it’s feature-gated and that the feature wasn’t set when generating the
documentation. An example with <a href="https://crates.io/crates/splines">splines</a> <a href="https://docs.rs/splines/0.2.3/splines/trait.Interpolate.html#foreign-impls">here</a>.
You can see a few implementors of the <code>Interpolate</code> trait (some basic types and some from <a href="https://crates.io/crates/cgmath">cgmath</a>
because the <code>&quot;impl-cgmath&quot;</code> feature is enabled by default) but not the potential ones for
<a href="https://crates.io/crates/nalgebra">nalgebra</a>, which is somehow misleading – people don’t know they can actually use <a href="https://crates.io/crates/nalgebra">nalgebra</a> with
<a href="https://crates.io/crates/splines">splines</a>!</p>
<p>A workaround to this problem is to get a local copy of the documentation, generating it with the
wished features. However, this will not fix the problem of the discoverability.</p>
<p>A solution to this could be to simply <em>“ignore”</em> the features while generating the documentation –
and only while generating it <strong>but annotate the documented symbols with the features.</strong> That would
enable something like this (generated with <code>cargo doc --open --no-default-features</code> and retouched
to mock the idea up):</p>
<p><img src="https://phaazon.net/media/uploads/documented_features.png" alt="" /></p>
<p>If several features are required, they would just be presented as a list above the annotated item.</p>
<p>That’s all for me today. Please provide your feedback about that idea. If you like it, I’ll write an
RFC because I really think it’s a missing thing in the documentation world of Rust.</p>
<p>Keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/luminance_sampleMon, 25 Jul 2016 00:00:00 GMT
title: luminance-0.6.0 sample
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance_sample
feedurl: https://phaazon.net/blog/feed
content:
<p>It’s been two weeks <a href="https://crates.io/crates/luminance">luminance</a> is at version 0.6.0. I’m very
busy currently but I decided to put some effort into making a very minimalistic yet usable sample.
The sample uses <a href="https://crates.io/crates/luminance">luminance</a> and
<a href="https://crates.io/crates/luminance-gl">luminance-gl</a> (the <strong>OpenGL 3.3</strong> backend being the single
one available for now).</p>
<p>You’ll find it <a href="https://github.com/phaazon/luminance-samples-rs/blob/master/src/main.rs">here</a>.
The code is heavily commented and you can of course clone the repository and and run the executable
with cargo.</p>
<p>I’ll post a more detailed blog post about the application I’m building with luminance right now
later on.</p>
<p>Keep the vibe! :)</p>


guid: https://strongly-typed-thoughts.net/blog/luminance_vertex_arraysMon, 10 Aug 2015 00:00:00 GMT
title: Luminance – Vertex Arrays
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance_vertex_arrays
feedurl: https://phaazon.net/blog/feed
content:
<p>I’ve been up working on vertex arrays in my work-in-progress graphics
framework, <a href="https://github.com/phaazon/luminance">luminance</a>, for several
days. I’m a bit slow, because I’ve been through a very hard breakup and have
been struggling to recover and focus. But here I am!</p>
<h1>So, what’s new?</h1>
<p><strong>OpenGL</strong> allows programmers to send <em>vertices</em> to the GPU through what
is called a <a href="https://www.opengl.org/wiki/Vertex_Specification">vertex array</a>.
Vertex specification is performed through several functions, operating on
several objects. You need, for instance, a <em>vertex buffer object</em>, an <em>index
buffer object</em> and a <em>vertex array object</em>. The <em>vertex buffer</em> stores the
vertices data.</p>
<p><img src="http://wiki.splashdamage.com/upload/2/2f/Teapot_mesh.jpg" alt="Teapot" /></p>
<p>For instance, you could imagine a <em>teapot</em> as a set of vertices. Those
vertices have several attributes. We could use, for instance, a <strong>position</strong>, a
<strong>normal</strong> and a <strong>bone index</strong>. The vertex buffer would be responsible of storing those
positions, normals and bone indices. There’re two ways to store them:</p>
<ol>
<li>interleaved arrays ;</li>
<li>deinterleaved arrays.</li>
</ol>
<p>I’ll explain those later on. The <em>index buffer</em> stores integral numbers –
mainly set to <code>unsigned int</code> – that index the vertices, so that we can connect
them and create lines, triangles or more complex shapes.</p>
<p>Finally, the <em>vertex array object</em> is a state object that stores links to the
two buffers and makes a connection between pointers in the buffer and attribute
indices. Once everything is set up, we might only use the <em>vertex array object</em>.
The exception is when we need to change the geometry of an object. We need to
access the vertex buffer and the index buffer and upload new data. However,
<strong>for now</strong>, that feature is disabled so that the buffers are not exposed to the
programmer. If people think that feature should be implemented, I’ll create
specialized code for that very purpose.</p>
<h2>Interleaved and deinterleaved arrays</h2>
<p>Interleaved arrays might be the most simple to picture, because you use such
arrays every day when programming. Let’s imagine you have the following type in
<strong>Haskell</strong>:</p>
<pre><code class="language-haskell">data Vertex = Vertex {
    vertPos    :: X
  , vertNor    :: Y
  , vertBoneID :: Z
  } deriving (Eq,Show)
</code></pre>
<p>Now, the teapot would have several vertices. Approximately, let’s state the
teapot has five vertices – yeah, ugly teapot. We can represent such vertices
in an interleaved array by simply recording them in a list or an array:</p>
<p><img src="http://phaazon.net/pub/interleaved.png" alt="Interleaved" /></p>
<p>As you can see, the attributes are interleaved in memory, and the whole pattern
is cycling. That’s the common way to represent an array of struct in a lot of
languages, and it’s very natural for a machine to do things like that.</p>
<p>The deinterleaved version is:</p>
<p><img src="http://phaazon.net/pub/deinterleaved.png" alt="Deinterleaved" /></p>
<p>As you can see, with deinterleaved arrays, all attributes are extracted and
grouped. If you want to access the third vertex, you need to read the third <code>X</code>,
the third <code>Y</code> and the third <code>Z</code>.</p>
<p>Both the methods have advantages and drawbacks. The cool thing about
deinterleaved arrays is that we can copy huge regions of typed memory at once
whilst we cannot with interleaved arrays. However, interleaved arrays store
continuous structures, so writing and reading a structure back might be faster.</p>
<p>An important point to keep in mind: because we plan to pass those arrays to
<strong>OpenGL</strong>, there’s no
<a href="https://en.wikipedia.org/wiki/Data_structure_alignment">alignment</a> restriction
on the structure. That is, everything is <em>packed</em>, and we’ll have to pass extra
information to <strong>OpenGL</strong> to tell it how to advance in memory to correctly
build vertices back.</p>
<h2>Generalized tuple</h2>
<p>I think I haven’t told you yet. I have a cool type in
<a href="https://github.com/phaazon/luminance">luminance</a>: the <code>(:.)</code> type. No, you
don’t have to know how to pronounce that. I like to call it the <em>gtuple</em>
type, because it’s a generalized tuple. You can encode <code>(a,b)</code>, <code>(a,b,c)</code> and
all kind of tuples with <code>(:.)</code>. You can even encode single-typed infinite
tuple! – a very special kind of list, indeed.</p>
<pre><code class="language-haskell">data a :. b = a :. b

infixr 6 :.

-- a :. b is isomorphic to (a,b)
-- a :. b :. c is isomorphic to (a,b,c)

newtype Fix f = Fix (f (Fix f)) -- from Control.Monad.Fix
type Inf a = Fix ((:.) a) -- infinite tuple!
</code></pre>
<p>Pretty simple, but way more powerful than the regular, monomorphic tuples. As
you can see, <code>(:.)</code> is a right-associative. That means that
<code>a :. b :. c = a :. (b :. c)</code>.</p>
<p>That type will be heavily used in
<a href="https://github.com/phaazon/luminance">luminance</a>, thus you should get your fet
wet with it. There’s actually nothing much to know about it. It’s a <code>Functor</code>.
I might add other features to it later on.</p>
<h3>The Storable trick</h3>
<p>The cool thing about <code>(:.)</code> is that we can provide a <code>Storable</code> instance for
packed memory, as <strong>OpenGL</strong> requires it. Currently, the <code>Storable</code> instance
is <a href="https://github.com/phaazon/luminance/blob/05ef2e4879aae92189535f0121765931b20de2fd/src/Graphics/Luminance/Tuple.hs#L23">implemented like this</a>:</p>
<pre><code class="language-haskell">instance (Storable a,Storable b) =&gt; Storable (a :. b) where
  sizeOf (a :. b) = sizeOf a + sizeOf b
  alignment _ = 1 -- packed data
  peek p = do
    a &lt;- peek $ castPtr p
    b &lt;- peek . castPtr $ p `plusPtr` sizeOf (undefined :: a)
    pure $ a :. b
  poke p (a :. b) = do
    poke (castPtr p) a
    poke (castPtr $ p `plusPtr` sizeOf (undefined :: a)) b
</code></pre>
<p>As you can see, the <code>alignment</code> is set to <code>1</code> to express the fact the memory
is packed. The <code>peek</code> and <code>poke</code> functions use the size of the head of the tuple
to advance the pointer so that we effectively write the whole tuple in packed
memory.</p>
<p>Then, let’s rewrite our <code>Vertex</code> type in terms of <code>(:.)</code> to see how it’s going
on:</p>
<pre><code class="language-haskell">type Vertex = X :. Y :. Z
</code></pre>
<p>If <code>X</code>, <code>Y</code> and <code>Z</code> are in <code>Storable</code>, we can directly <code>poke</code> one of our
<code>Vertex</code> into a <a href="http://phaazon.blogspot.fr/2015/07/introducing-luminance-safer-opengl-api.html">luminance buffer</a>! That is, directly into the GPU buffer!</p>
<p>Keep in mind that the <code>Storable</code> instance implements packed-memory uploads and
reads, and won’t work with special kinds of buffers, like <em>shader storage</em> ones,
which require specific memory alignment. To cover them, I’ll create specific
typeclasses instances. No worries.</p>
<h1>Creating a vertex array</h1>
<p>Creating a vertex array is done through the function <code>createVertexArray</code>. I
might change the name of that object – it’s ugly, right? Maybe <code>Shape</code>, or
something cooler!</p>
<pre><code class="language-haskell">createVertexArray :: (Foldable f,MonadIO m,MonadResource m,Storable v,Traversable t,Vertex v)
                  =&gt; t v
                  -&gt; f Word32
                  -&gt; m VertexArray
</code></pre>
<p>As you can see, the type signature is highly polymorphic. <code>t</code> and <code>f</code> represent
<em>foldable</em> structures storing the vertices and the indices. And that’s all.
Nothing else to feed the function with! As you can see, there’s a typeclass
constraint on <code>v</code>, the inner vertex type, <code>Vertex</code>. That constraint ensures the
vertex type is representable on the <strong>OpenGL</strong> side and has a known vertex
format.</p>
<p><strong>Disclaimer:</strong> the <code>Traversable</code> constraint might be relaxed to be <code>Foldable</code>
very soon.</p>
<p>Once tested, I’ll move all that code from the <code>unstable</code> branch to the <code>master</code>
branch so that you guys can test it. :)</p>
<h2>About OpenGL…</h2>
<p>I eventually came to the realization that I needed to inform you about the
<strong>OpenGL</strong> prerequisites. Because I want the framework to be as modern and
well-designed as possible, you’ll need… <strong>OpenGL 4.5</strong>. The latest version,
indeed. You <strong>might</strong> also need an extension, <a href="https://www.opengl.org/wiki/Bindless_Texture">ARB_bindless_texture</a>. That would enable the framework to pass textures to
shader in a very stateless way, which is our objective!</p>
<p>I’ll let you know what I decide about that. I don’t want to use an extension
that is not implemented almost everywhere.</p>
<h1>What’s next?</h1>
<p>Well, tests! I need to be sure everything is correctly done on the GPU side,
especially the vertex format specification. I’m pretty confident though.</p>
<p>Once the vertex arrays are tested, I’ll start defining a <em>render interface</em> as
stateless as I can. As always, I’ll keep you informed!</p>


guid: https://strongly-typed-thoughts.net/blog/c++-exceptionsSat, 29 Feb 2020 20:20:00 GMT
title: Let’s talk about C++ exceptions
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/c++-exceptions
feedurl: https://phaazon.net/blog/feed
content:
<blockquote>
<p>This article is a sequel to <a href="https://phaazon.net/blog/c++-constructors">Let’s talk about C++ constructors</a>.
Even though not required, reading that article first might give you more information on the motivation and
my state of mind regarding C++ and software development.</p>
</blockquote>
<p>This time, I want to talk about C++ exceptions. But talking about exceptions for the sake of talking about
them has very little value. Let’s instead talk about error handling.</p>
<h1>Infallible pure functions</h1>
<p>Imagine a function that takes some input, like a <code>String</code>, and outputs something else, like the length of the
string. Such a function is <em>pure</em>: its output depends <em>only</em> on its arguments. It cannot have any side-effect
nor return something else. It is also infallible as it can <em>always</em> return a valid length for all possible
values it can be called with. Imagine the following C++ implementation:</p>
<pre><code class="language-cpp">size_t get_len(std::string const &amp; s) {
  return s.length();
}
</code></pre>
<blockquote>
<p>Even though C++ doesn’t have the concept of purity, let’s assume we can express the concept, just because
it’s a powerful concept that we can apply to any language. The compiler will not know about it, but the way
we design things does.</p>
</blockquote>
<p>That function is pure and infallible. When you look at the signature, you see that it takes a read-only
reference on a <code>std::string</code> , and returns a <code>size_t</code>. And yes, I know about <code>noexcept</code>. However, this
specifier cannot be trusted. Consider:</p>
<pre><code class="language-cpp">#include &lt;string&gt;

void foo() {
  throw std::runtime_error(&quot;That should be seen by the compiler, right?&quot;);
}

size_t get_len(std::string const &amp; s) noexcept {
  foo();
  return s.length();
}

int main() {
  std::string foo = &quot;foo&quot;;
  get_len(foo);
  return 0;
}
</code></pre>
<p>Compile with (I’m running Archlinux):</p>
<pre><code>g++ -std=c++2a -Wall -pedantic a.cpp
</code></pre>
<p>No compiler warning. Run it:</p>
<pre><code>terminate called after throwing an instance of 'std::runtime_error'
  what():  That should be seen by the compiler, right?
zsh: abort (core dumped)  ./a.out
</code></pre>
<p>What that little snippet tells us is that even though <code>get_len</code> is annotated with <code>noexcept</code>… it can still
throw. Not directly in its body, but functions it calls may throw. When I was introduced to that keyword,
years ago, I was suspicious. Since C++ will throw exceptions for — erm — exceptional errors, such as
<em>out of memory</em> errors, then… even a <code>noexcept</code> function can still throw errors. Then, because of that,
<code>noexcept</code> cannot propagate downwards in your call stack. If <code>A</code> is <code>noexcept</code> and <code>B</code> <em>may throw</em>, then
calling <code>B</code> from <code>A</code> is valid in C++.</p>
<p><code>noexcept</code> is just a documentation keyword and an opportunity for your compiler to optimize your code. Your
compiler can only emit warnings if you use the <code>throw</code> keyword directly in the body of the function that is
marked <code>noexcept</code>. Also, it’s important to notice that, given the team you work in, or the project you work
on, it’s possible to see the use of <code>noexcept</code>… like not at all. It’s all about convention; your compiler
will not enforce that… which is a problem to me. It’s a problem because more freedom means more opportunities
for people to make mistake. To forget about annotating a function with <code>noexcept</code>. Or, worse, it gives
opportunities to people who just don’t care and want to rush, making reviewing their code more challenging
than needed.</p>
<h1>Fallible functions</h1>
<p>In my previous article, I’ve been criticized for not explaining enough what I mean about <em>exceptional
errors</em> and that it was a highly subjective point. I’ll try to explain more in this section.</p>
<p>Imagine that you want to implement a function that will perform a lookup. If it finds the key you give
as argument, it will return the associated object. However, what to do when the key is not there? If you read
a bit my previous article, you know that I would use a sum type to encode the error in the type system. But
let’s do it <em>the C++ way</em>. Let’s use exceptions.</p>
<pre><code class="language-cpp">Object lookup(Key const &amp; key);
</code></pre>
<p>If you look at that signature, you’ll notice an important point. There is no error handling annotation. Most
of the time, people will follow some guidelines to put that information in the documentation directly.
However, several points:</p>
<ul>
<li>What happens if a teammate of yours or even yourself – after some weeks / months – forgets about that
documentation line?</li>
<li>Because you might call that function from <em>anywhere</em> — even in <code>noexcept</code> code, as demonstrated in the
previous section — how do you know, when reading the code, that a call to this function can throw?</li>
<li>The last point is especially true when refactoring. Imagine that this function belongs to a block of code
delimited with a <code>try catch</code> block. Do you assume the whole block as atomic? If so, do you lookup for the
documentation of <em>all of the functions</em> called in that block? What happens if you move that function out
of the block?</li>
</ul>
<p>Now, that just assumes a flat block. But it’s easy to guess that you will have to do that for the whole stack
of functions being called — i.e. as soon as you find a <code>noexcept</code> function in the stack… well nothing, you
have to go on, since a <code>throw</code> might be hidden in a function deeper in the stack.</p>
<p>Most of the time, the replies I get about that topic are, either:</p>
<ul>
<li><strong><em>“Just read the documentation”</em></strong>. That argument completely ignores the last part of my point above — i.e. do you
<em>really</em> read the documentation of <em>all</em> the functions being directly or transitively called in a
function? That seems insane. Also, you might argue that the documentation can say, at a function level <code>N</code>,
that there is a throw at level <code>N - k</code>. However, that seems like an impossible task to maintain. You might
forget to update the documentation if you stop throwing that exception or throw another object with a
different type, etc.</li>
<li><strong><em>“Use better names. If you use a function that, in its name, expresses the idea that it might fail, it’s
easier to refactor / use the function correctly”</em></strong>. That is true, even though you will <em>always</em> find someone
abusing it and using it without a <code>try catch</code> block.</li>
<li><strong><em>“We don’t care about handling errors: we will just handle them at the top-level of the program / function
call stack with a single try catch block”</em></strong>. That argument doesn’t survive five seconds as soon as you
talk about, for instance, serialization or map lookups.</li>
</ul>
<p>About the documentation and naming… it adds another problem: humans. We are fallible. You might work on a
project that doesn’t document correctly. Or that doesn’t even have proper convention. Or several ones. When
considering exceptions for error handling, I think it’s important to imagine what will happen in X months.
After the codebase has become complex, large, with a lot of edge cases and possible errors. Maintainability
should be a goal. No one enjoys having to read through the bodies of twenty functions to understand why their
program crashed or why the GUI displays a pop-up with the content of an exception.</p>
<p>The sooner you can see an error, the better. If that <em>sooner</em> can be “compile-time”… why would you want to
still push the error to the runtime? There are things I will never understand.</p>
<p>On the other side, consider sum types:</p>
<pre><code class="language-cpp">std::optional&lt;Object&gt; lookup(Key const &amp; key);
</code></pre>
<p>Even though it’s still pretty bad to me because of how <code>std::optional</code> is made (have a look at my previous
article), it has the advantage of being typed. No documentation is needed — but please do document to
explain what can fail though — and your compiler can safely prevent you from doing bad things. Of course,
this is limited by how you use the <code>std::optional</code>, as C++ doesn’t have pattern-matching. But I would like
to reply to an argument I hear every now and then: <strong>it’s not because a better solution is not perfect that
it should be discarded to stay on your legacy solution</strong>. Imagine that you have a tool <code>X</code> with several
issues, <code>{a, b, c, d}</code>. Now imagine we suggest to switch to a new tool, <code>Y</code>, with issues <code>{a, d}</code> only.
Yes, you still have two issues… but you have less. In the case of exceptions vs. a type-system, in the
case of C++, yes, you can still call <code>.value()</code> on an empty <code>std::optional</code> and crash your program. But
you don’t have the problem of hidden and untracked error handling. You can simply use exceptions for
exceptional cases. Those cases that are <em>not</em> function boundaries nor edge cases. And yes, I do think that
most of the standard C++ exceptions, such as <code>std::invalid_argument</code>, are to be completely avoided.</p>
<p>But here, exceptions have an advantage if we stop there: they provide an error description.</p>
<h1>Fully-typed failures</h1>
<p>Fixing that problem is pretty trivial with a strong type-system and
<a href="https://en.wikipedia.org/wiki/Algebraic_data_type">algebraic data types</a>. We want to use those to create
a <em>result</em> type, that can either be something, or a typed error, that would contain exactly the same
information you have in a regular exception.</p>
<p>C++ doesn’t really have that out of the box but it could be made, in the same way <code>std::variant</code> exists.
Imagine a hypothetical <code>std::either</code> type and let’s implement a function that parses something:</p>
<pre><code class="language-cpp">std::either&lt;Something, ParseError&gt; parse(std::string const &amp; input);
</code></pre>
<p>That function returns <em>either</em> a successful object (<code>Something</code>) or an error (<code>ParseError</code>).</p>
<p>With that signature, it’s clear that the function can fail with <code>ParseError</code>. The point is that the caller
<em>must</em> do something about it. If they don’t know what to do with it — imagine a parser combinator or some
code that doesn’t know how to handle a parse error and <em>requires</em> its caller to handle the error — then
the function needs to abort and propagate the error upwards. That looks like a bit like the interface you
have with exceptions… but here, the interface is visible at the type-level.</p>
<p>Obviously, you cannot use <code>throw</code> to propagate upwards. You need to use <code>return</code> from your function. With
either a macro or some language construct, C++ could make that propagation smoother, but currently, it
doesn’t have a proper way to do it. So we’d be left with macros only, or manual propagation. Since C++
doesn’t have pattern-matching nor exhaustive patterns, it would be pretty hard to implement that mechanism
in a complete sound way. As with <code>std::optional</code>, it’s not perfect, but it would be slightly better than
using opaque exceptions.</p>
<p>One final point. Sometimes, I wonder what it would be like to just give up on my ideas of using a strong
type system in C++. The language is using exceptions and people are used to it. That’s <em>the C++ way</em>. So…
why not changing the way exceptions work so that they’re not opaque, and propagate upwards? I remember
the <code>throw</code> specifier, used to state what a function might throw. But again, it’s not enforced by compiler.
Worse, it’s being deprecated in C++ 2020.</p>
<p>I voluntarily omitted any reference to either Haskell, Rust or any language like that so that people don’t
think I’m trying to compare C++ to another language. I’m having a look on C++ after almost two decades using
it and what else I’ve learned. The situation is, to me, frustrating. Because yes, whatever the good arguments
against exceptions, people still use them and error handling in C++ is still about exceptions. So you still
can have constructors that fail. You still depend a lot on documentation and your compiler cannot tell you
when something is not okay. We are all fallible, way more than we think.</p>


guid: https://strongly-typed-thoughts.net/blog/proc-macro-faithful-displaySat, 01 Dec 2018 19:51:00 GMT
title: A more faithful Display for proc-macro token types
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/proc-macro-faithful-display
feedurl: https://phaazon.net/blog/feed
content:
<blockquote>
<p>This article serves as an announcement to a new crate: <a href="https://crates.io/crates/proc-macro-faithful-display">proc-macro-faithful-display</a>.</p>
</blockquote>
<h1>Enhancing faith in displaying Rust tokens</h1>
<h2>The problem</h2>
<p>I’ve been around procedural macros in Rust for a few weeks now – in order to write my
<a href="https://crates.io/crates/glsl-quasiquote">glsl-quasiquote</a> crate. Quickly, I discovered that most types cannot be inspected. For instance,
an <a href="https://doc.rust-lang.org/stable/proc_macro/struct.Ident.html"><code>Ident</code></a> has no methods to get the underlying identifier (as a <code>&amp;str</code> for instance). The only
way to do so is to allocate a <code>String</code>, for instance, via a <code>Display</code> use, as in:</p>
<pre><code>let ident_str = format!(&quot;{}&quot;, ident);
</code></pre>
<p>Now, imagine you have a fully formatted stream of GLSL tokens flowing in.
<a href="/blog/glsl-quasiquote-0.2">I’ve already blogged</a>
lately about the fact Rust has some specific rules that will prevent you to write <em>preprocessor
pragmas</em>. The only way to authorize them was to implement a hack (explained in the blog article).</p>
<p>Lately, I was working on some not-so-trivial glsl transformations in another project when I hit a
parse error. I was surprised, because the <a href="https://crates.io/crates/glsl">glsl</a> crate contains more than 130 unit tests about
parsing. So I thought <em>“Well, okay, it’s alright, I might have forgotten a very specific case not
covered in my nom implementation. I’ll just get a small reproducible sample and use it as a
new integration test in <a href="https://crates.io/crates/glsl">glsl</a>”</em>. That kind of reasoning is very important to me: if you find a
bug, first thing to do (if you plan to fix it) is to write a unit test <strong>that fails first.</strong>
This is very important because when the test succeeds, you have solved your problem. If someone
provides a contribution later and that the test breaks again, you will have just shielded the
project against a test regression. Seriously, I know it’s boring but: <strong>write tests.</strong></p>
<p>So I wrote a test, a
<a href="https://github.com/phaazon/glsl/blob/ec7be696a53e3b7b12ff6ccf72c4fcf3465a1840/glsl/tests/missing_zero_float.rs#L7">very simple one</a>
(this is actually an integration test, since I’ve found the error in a project using <a href="https://crates.io/crates/glsl">glsl</a>). And…
the test passed. I was so surprised: <em>“So that code breaks in my crate, but succeeds in the
integration test suite?! Well, wait. That means that the parser is actually correct. The problem
might come from the way the parser is used… or the input.”</em></p>
<p>So my second reaction was to have a look at where the parser was called. It was in a <code>glsl!</code>
procedural macro invocation – in my project, I hardcoded a <em>vertex shader</em> with <a href="https://crates.io/crates/glsl-quasiquote">glsl-quasiquote</a>.
And then, I came to the realization and remembered. <a href="https://crates.io/crates/glsl-quasiquote">glsl-quasiquote</a>, in order to parse the 
input Rust token, has to use <code>Display</code> on them, in order to yield a <code>String</code> usable by the <a href="https://crates.io/crates/glsl">glsl</a>
parser. The parser is okay – i.e. the integration test passes – but the input string… loses
information.</p>
<p>If you have taken a look at the integration test just above, you should have noticed an interesting
construction:</p>
<pre><code>void main() {
  float x = 1. * .5;
}
</code></pre>
<p>This is not a <em>semantically</em> valid vertex shader, but it should parse. And it does parse. But if
you write that vertex shader with <a href="https://crates.io/crates/glsl-quasiquote">glsl-quasiquote</a>, what you actually write is this:</p>
<pre><code>let vertex_shader = glsl!{
  void main() {
    float x = 1. * .5;
  }
};
</code></pre>
<p>The Rust tokenizer will eat the tokens and the <code>Display</code> implementation will render the stream
by using spaces whenever it sees fit. It will not respect the initial whitespaces. It might yield
something like this:</p>
<pre><code>void main ( ) { float x = 1. * . 5 ; }
</code></pre>
<p>Look closely: our <code>.5</code> GLSL floating point constant value (it equals <code>0.5</code>) has become <code>.</code> and <code>5</code>.
<strong>rustc</strong> has transformed it into two tokens instead of a single one. This is due to the fact that
this float format is not allowed in Rust and is interpreted the same way a method call would be.</p>
<h2>The solution</h2>
<p>So, after realizing that, I just went for a documentation addition stating that numbers cannot be
expressed this way, and that you have to use a leading <code>0</code>. I really disliked writing that
documentation exception but I didn’t have any solution to that.</p>
<p>A few days later, I was thinking about <a href="https://crates.io/crates/glsl">glsl</a> variable interpolation and got a mind click. In order
to implement preprocessor pragmas, I used a trick with <a href="https://doc.rust-lang.org/stable/proc_macro/struct.Span.html"><code>Span</code></a>, that gave me positional information
of tokens. And I came to the realization that I could use the same kind of trick to implement a
function that would yield an object implementing <code>Display</code> by respecting the initial
layout / formatting!</p>
<p>Basically, a trait – <a href="https://docs.rs/proc-macro-faithful-display/0.1.0/proc_macro_faithful_display/trait.FaithfulDisplay.html"><code>FaithfulDisplay</code></a> – is implemented for all flavours of Rust tokens:</p>
<ul>
<li><a href="https://doc.rust-lang.org/stable/proc_macro/struct.Ident.html"><code>Ident</code></a>.</li>
<li><a href="https://doc.rust-lang.org/stable/proc_macro/struct.Literal.html"><code>Literal</code></a>.</li>
<li><a href="https://doc.rust-lang.org/stable/proc_macro/struct.Punct.html"><code>Punct</code></a>.</li>
<li><a href="https://doc.rust-lang.org/stable/proc_macro/struct.Group.html"><code>Group</code></a>.</li>
<li><a href="https://doc.rust-lang.org/stable/proc_macro/struct.TokenTree.html"><code>TokenTree</code></a>.</li>
<li><a href="https://doc.rust-lang.org/stable/proc_macro/struct.TokenStream.html"><code>TokenStream</code></a>.</li>
</ul>
<p>All the implementations carefuly carry around the previous <a href="https://doc.rust-lang.org/stable/proc_macro/struct.Span.html"><code>Span</code></a> in order to <em>pad</em> enough spaces
and newlines before they get formatted into the <code>Formatter</code>. This allows to reconstruct the input
layout.</p>
<p>All this code is packaged in the <a href="https://crates.io/crates/proc-macro-faithful-display">proc-macro-faithful-display</a> crate. You can find the
documentation <a href="https://docs.rs/proc-macro-faithful-display">here</a>. I use it – and tested against –
the crate with <a href="https://crates.io/crates/glsl-quasiquote">glsl-quasiquote</a>. It especially allowed me to remove all the preprocessor trick
because now newlines are taken into account.</p>
<h2>Is this viable?</h2>
<p>Honestly, I have no idea. Currently, at the time of writing, it works like a charm. If you find any
bug or weird behavior, please <a href="https://github.com/phaazon/glsl/issues">open an issue here</a>.
However, <a href="https://doc.rust-lang.org/stable/proc_macro"><code>proc-macro</code></a> hasn’t yet been stabilized and the <a href="https://doc.rust-lang.org/stable/proc_macro/struct.Span.html"><code>Span</code></a> type is accessible through a
<em>feature gate</em>. All of this might change. However, I strongly think that:</p>
<ul>
<li>Either Rust will keep that <a href="https://doc.rust-lang.org/stable/proc_macro/struct.Span.html"><code>Span</code></a> type because it is really helpful (its first purpose was to    provide nice error reporting).</li>
<li>Either they’ll ditch it, but since quasiquoting is important and wanted (have a look at
<a href="https://github.com/bodil">@bodil</a>’s awesome <a href="https://github.com/bodil/typed-html">typed-html</a> crate for instance), something to replace it might come next.</li>
</ul>
<p>Keep the vibes!</p>


title: Abstracting shader – Environment
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/abstracting_over_shader_environment
feedurl: https://phaazon.net/blog/feed
guid:
https://strongly-typed-thoughts.net/blog/abstracting_over_shader_environmentMon, 17 Nov 2014 00:00:00 GMT

content:
<h1>In the previous episode…</h1>
<p>This blog entry directly follows the one in which I introduced <strong>Ash</strong>, a <em>shading language</em> embedded in <strong>Haskell</strong>. Feel free to <a href="http://phaazon.blogspot.co.uk/2014/11/foreword-abstracting-what-shaders-are.html">read it here</a> before going on.</p>
<h1>Controlling behavior</h1>
<p>A <em>shader</em> is commonly a function. However, it’s a bit more than a simple function. If you’re a haskeller, you might already know the <code>MonadReader</code> typeclass or simply <code>Reader</code> (or its transformer version <code>ReaderT</code>). Well, a shader is kind of a function in a reader monad.</p>
<blockquote>
<p><em>So… that implies a shader runs in… an environment?</em></p>
</blockquote>
<p>Yeah, exactly! And you define that environment. The environment is guaranteed not to change between two invocations of a shader for the same render (e.g. between two vertices in the same render). This is interesting, because it enables you to use nice variables, such as time, screen resolution, matrices and whatever your imagination brings on ;)</p>
<p>The environment, however, can be changed between two renders, so that you can update the time value passed to the shader, the new resolution if the window resizes, the updated matrices since your camera’s moved, and so on and so forth.</p>
<p>Let’s see a few example in GLSL first.</p>
<h2>Shader environment in GLSL</h2>
<p>To control the environment of a shader in GLSL, we use <em>uniform</em> variables. Those are special, global variables and shared between all stages of a shader chain[^shader_chain].</p>
<p>Let’s see how to introduce a few <em>uniforms</em> in our shader:</p>
<pre><code>uniform float time;       // time of the host application
uniform vec2 resolution;  // (w,h)
uniform vec2 iresolution; // (1 / w, 1 / h), really useful in a lot of cases ;)
uniform mat4 proj;        // projection matrix
uniform int seed;         // a seed for whatever purpose (perlin noise?)
uniform ivec2 gridSize;   // size of a perlin noise grid!
</code></pre>
<p>You got it. Nothing fancy. Those uniforms are shared between all stages so that we can use <code>time</code> in all our shaders, which is pretty cool. You use them as any kind of other variables.</p>
<p>Ok, let’s write an expression that takes a time, a bias value, and multiply them between each other:</p>
<pre><code>uniform float time;
uniform float bias;

// this is not a valid shader, just the expression using it
time * bias;
</code></pre>
<h2>Shader environment in HLSL</h2>
<p><strong>HLSL</strong> uses the term <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/ff476896(v=vs.85).aspx">constant buffers</a> to introduce the concept of environment. I don’t have any examples right now, sorry for the inconvenience.</p>
<h2>Shader environment in Ash</h2>
<p>In Ash, environment variables are not called uniforms nor constant buffers. They’re called… <em>CPU variables</em>. That might be weird at first, but let’s think of it. Those values are handled through your application, which lives CPU-side. The environment is like a bridge between the CPU world and the GPU one. A CPU variable refers to a constant value GPU-side, but varying CPU-side.</p>
<p>Create a <em>CPU variable</em> is pretty straight-forward. You have to use a function called <code>cpu</code>. That function is a monadic function working in the EDSL monad. I won’t describe that type since it’s still a work in progress, but it’s a monad for sure.</p>
<blockquote>
<p><strong>Note: If you’ve read the <a href="http://phaazon.blogspot.co.uk/2014/11/foreword-abstracting-what-shaders-are.html">previous blog entry</a>, you might have come across the <code>Ash</code> type, describing a HOAST. That type is no more a HOAST. The “new Ash” – the type describing the HOAST – is now Expr.</strong></p>
</blockquote>
<p>This is <code>cpu</code>:</p>
<pre><code>cpu :: (CPU a) =&gt; Chain (Expr a)
</code></pre>
<p><code>CPU</code> is a typeclass that enables a type to be injected in the environment of a shader chain. The instances are provided by Ash and you can’t make your own – do you really want to make <code>instance CPU String</code>, or <code>instance (CPU a) =&gt; CPU (Maybe a)</code>? Don’t think so ;)</p>
<p>Let’s implement the same time–bias example as the GLSL one:</p>
<pre><code>foo :: Chain (Expr Float)
foo = liftA2 (*) cpu cpu
</code></pre>
<p>That example is ridiculous, since in normal code, you’d actually want to pass the CPU variables to nested expressions, in shaders. So you could do that:</p>
<pre><code>foo :: Chain ()
foo = do
  time &lt;- cpu
  bias &lt;- cpu

  -- do whatever you want with time and bias
  return ()
</code></pre>
<h1>You said Chain?</h1>
<p><code>Chain</code> is a new type I introduce in this paper. The idea came up from a discussion I had with Edward Kmett when I discovered that the EDSL <strong>needed</strong> a way to bind the CPU variables. I spotted two ways to go:</p>
<ul>
<li>using a name, like <code>String</code>, passed to <code>cpu</code>; that would result in writing the name in every shader using it, so that’s not ideal;</li>
<li>introducing the environment and providing a monad instance so that we could bind the CPU variables and use them in shaders inside the monad.</li>
</ul>
<p>The latter also provides a nice hidden feature. A chain of shaders might imply varying[^varying] values. Those varying values have information attached. If you mistake them, that results in catastrophic situations. Using a higher monadic type to capture that information – along with the environment – is in my opinion pretty great because it can prevent you from going into the wall ;).</p>
<p>To sum up, <code>Chain</code> provides a clear way to describe the relation between shaders.</p>
<h1>What’s next?</h1>
<p>I’m still building and enhancing Ash. In the next post, I’ll try to detail the interface to build functions, but I still need to find how to represent them the best possible way.</p>
<p>[^shader_chain]: You can imagine a shader chain as an explicit composition of functions (i.e. shaders). For instance, a vertex shader followed by geometry shader, itself followed by a fragment shader.
[^varying]: Varying values are values that travel between shaders. When a shader outputs a value, it can go to the input of another shader. That is call a varying value.</p>


guid: https://strongly-typed-thoughts.net/blog/glsl-0.13-visitorThu, 22 Nov 2018 04:20:00 GMT
title: glsl-0.13 and its visitor pattern
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/glsl-0.13-visitor
feedurl: https://phaazon.net/blog/feed
content:
<blockquote>
<p>This article serves as an announcement for <a href="https://crates.io/crates/glsl/0.13.2">glsl-0.13</a>.</p>
</blockquote>
<p>I’ve been working on <a href="https://crates.io/crates/glsl/0.13.2">glsl-0.13</a> for a few days now since my <a href="https://phaazon.net/blog/glsl-pest-part-2">previous blog entry</a> about adding
<a href="https://crates.io/crates/pest">pest</a> to <a href="https://crates.io/crates/glsl">glsl</a> – and no, the current code is not pest-based, it’s still good old <a href="https://crates.io/crates/nom">nom</a>. Lately,
I’ve been wanting to enhance my <a href="https://crates.io/crates/glsl-quasiquote">glsl-quasiquote</a> crate to add <a href="https://en.wikipedia.org/wiki/String_interpolation">variable interpolation</a>. I think I
will dedicate a whole article to <em>variable interpolation</em> in GLSL because it’s actually tricky to
get done right – without duplicating a whole parser or introducing problematic code with pest (see
my last article).</p>
<p>Since I’m not a programmer who solves problems that don’t exist, I have problems to get resolved in
other crates and binary projects of mine (mostly demoscene and demoscene tooling). However, I’ve
been spending days solving those problems because they’re not related to demoscene only: pretty much
anyone who would like to cope with shaders might be interested.</p>
<h1>The <a href="https://crates.io/crates/glsl">glsl</a> crate prior to 0.13</h1>
<p>The <a href="https://crates.io/crates/glsl">glsl</a> crate provides you with:</p>
<ul>
<li>A GLSL450 parser. I wrote the parser with <a href="https://crates.io/crates/nom">nom</a> back then by implementing the strict OpenGL
Shading Language Spec (<a href="https://www.google.com/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=1&amp;cad=rja&amp;uact=8&amp;ved=2ahUKEwj3uZTS-ebeAhUuyoUKHcW1D04QFjAAegQIAhAC&amp;url=https%3A%2F%2Fwww.khronos.org%2Fregistry%2FOpenGL%2Fspecs%2Fgl%2FGLSLangSpec.4.50.pdf&amp;usg=AOvVaw01MeCZu7SMf3NwCmD83SAr">here</a>).
That parser works mostly on strings and byte slices (I really doubt people use the byte slice
interface and I might deprecate it).</li>
<li>The parser parses into an <a href="https://en.wikipedia.org/wiki/Abstract_syntax_tree">Abstract Syntax Tree</a> – AST for short. That AST is, as the name
implies, a tree that contains typed <em>nodes</em>. It’s a bit more complicated than having a
<code>BTreeMap</code> for instance, because each nodes and levels are completely typed. You can picture the
AST as both a <em>type tree</em> and <em>value tree</em> instead of just a <em>value tree</em>.</li>
<li>Before 0.13, transforming the AST was tedious, because you had to pattern match on the whole
structure of the AST, and since it’s a <em>type tree</em>, it’s really, really boring to do so.</li>
<li>One typical transformation that I always need in several crates of mine is <em>construction</em>.
Building ASTs was overwhelming and complicated because you had to create the tree by hand. To
solve that issue, I introduced the <a href="https://crates.io/crates/glsl-quasiquote">glsl-quasiquote</a> crate, that enables you to create the AST
out of a regular GLSL string. I hit some issues with that, like the impossibility to have typing
information exchanged between two compiler phases (the procedural macros are all evaluated
before the rest, making it impossible to have type inference like in <code>let x: Expr = glsl!{…};</code>.
I will try to fix that later.</li>
<li>The quasiquote crate works great for static ASTs but as soon as you want to build dynamic ones (
i.e. that depend on variables for instance), since <a href="https://crates.io/crates/glsl-quasiquote">glsl-quasiquote</a> doesn’t have variable
interpolation (yet!), you’re back to the boring by-hand construction.</li>
</ul>
<p><a href="https://crates.io/crates/glsl/0.13.2">glsl-0.13</a> comes with several new features and enhancements to partially fix those points that I
will explain in this blog entry.</p>
<h1>The new features!</h1>
<p>First, let’s talk about contribution. I received a contribution (both an issue <em>and</em> a PR, how
amazing!) from <a href="https://github.com/JDemler">@JDemler</a> about the <code>#define</code> preprocessor pragma. Those were not taken into
account so far and he provided the fix to add them! As for all other pragmas, those have to lay in
the global scope – they’re not expected to be found in function bodies, for instance.</p>
<p>Thanks for your contribution! :)</p>
<p>Then, a lot of work has been done around the <code>glsl::syntax</code> module. This module contains all the AST
types and some methods were added to, for instance, create <em>functions</em>, <em>variable declarations</em>,
<em>if-else</em> blocks, etc. Those functions were made the most general as possible and heavily depend on
<code>From</code> / <code>Into</code>, allowing for very short oneliners to create complex AST nodes. The best example
for this is the <a href="https://docs.rs/glsl/0.13.2/glsl/syntax/enum.Statement.html#method.declare_var"><code>Statement::declare_var</code></a> that declares a new variable as a <code>Statement</code>. Most of
the inner AST nodes won’t ever be needed in the public interface, so those functions hide them from
you and give you the easy way to build bigger nodes.</p>
<blockquote>
<p>There is an example below that uses <a href="https://docs.rs/glsl/0.13.2/glsl/syntax/enum.Statement.html#method.declare_var"><code>Statement::declare_var</code></a>. Keep on reading! :)</p>
</blockquote>
<p>Not all AST nodes have helper methods for construction. The good thing is that adding a new function
<em>shouldn’t</em> be a breaking change, so I’ll keep adding them as needed – if you have a specific need
for one, feel free to open an issue, I will make the change. That might be surprising but since I
haven’t witnessed lots of people using the crate yet, I don’t implement what I don’t need yet – but
as soon as someone tells me they need something, either I immediately review the PR, or plan some
time to make the patch myself.</p>
<p>Finally, the biggest feature which will be explained in further details in this blog post: <strong>AST
visitors</strong>. AST visitors are the way I imagined would be the best to traverse an AST in order to
mutate some nodes, all the nodes, filter, query information, etc. It’s like <em>lenses</em> for <a href="https://crates.io/crates/glsl">glsl</a>! –
but trust me: it’s way lighter! :D</p>
<h2>Visiting ASTs</h2>
<p>AST visitors solve a problem I had when implementing a specific feature in <a href="https://crates.io/crates/spectra">spectra</a>. I needed to be
able to change <strong>all</strong> the references to a given identifier in all the AST at once. Either the
identifier is used in an expression assigned to a new variable, returned from a function or passed
as argument to a function, I needed to catch that identifier… and change its name.</p>
<p>That was quite of a desperate challenge without a proper solution. Imagine: I would have to write
the code to change the identifier myself and… find ALL the places where identifiers <em>might</em> appear
in <em>any</em> AST. This is possible, but that would drive many developers mad – especially whevener the
<a href="https://crates.io/crates/glsl">glsl</a> crate changes.</p>
<p>So I came up with a better solution. I will not drop you a technical term and have you read
Wikipedia, I would dislike that. I will just explain from bottom up why it’s designed this way and
why I think it’s the best solution.</p>
<h2>Visiting summarized</h2>
<p>The idea is that pattern-matching on an identifier might appear in several places in an AST. So
whatever solution we choose, we will have to find all those spots and call a function that states:</p>
<blockquote>
<p><em>Hey there. Here is an <code>Identifier</code>. I give you a <code>&amp;mut Identifier</code> so that you can change it.</em></p>
</blockquote>
<p>An <code>Identifier</code> is an AST (a really simple one, but still is). So you might want to implement that
function on <code>Identifier</code>… but what happens when your AST is an <code>Expr</code>, and that expr is a variable –
that is, <code>Expr::Var(identifier_here)</code>? You will want to implement your function on <code>Expr</code> <em>also</em>,
then. And here you see that you need a trait, because, as said earlier, the AST is a <em>type tree</em>.</p>
<p>However, what trait? We could imagine implementing that trait only on AST types that have an
<code>Identifier</code> as field or variant. But the most general AST node, <code>TranslationUnit</code>, is a non-empty
list of <code>ExternalDeclaration</code>s. If we want to visit that, we won’t be able to type match and pass
down our function transforming identifiers.</p>
<p>We see that we will need to implement that trait for all AST types so that they can pass the
function down the AST to a node that actually has an <code>Identifier</code>.</p>
<p>And since we’re doing it with <code>Identifier</code>, we might want to do it with <em>any</em> AST node. But if we
do that, we cannot pass one single function anymore…</p>
<h2>The Visitor</h2>
<p>So you need an object that will be able to treat an <code>Identifier</code>… or a <code>TranslationUnit</code>, or
anything AST. This is a bit boring to implement, but we need a trait that enables a type to visit
any AST node:</p>
<pre><code>trait Visitor {
  fn visit_identifier(&amp;mut self, _: &amp;mut Identifier);
  fn visit_translation_unit(&amp;mut self, _: &amp;mut TranslationUnit);
  // …
}
</code></pre>
<p>This is great, because if a type implements <code>Visitor</code>, it means that we can call <code>visit_identifier</code>
on it if we have an <code>Identifier</code>! When we will be implementing our traversing function, we will just
have to carry around a mutable reference to an object implementing <code>Visitor</code>, and call the right
function depending on the AST node / type we are at!</p>
<p>We also use mutable references here so that we can also mutate information as we sink into the AST.
This might be very useful to know at which block depth we are at, or if we’re in a function’s body,
etc.</p>
<p>Something important with that trait though: the current implementation (from this article) would be
very boring to implement, because it has a lot of <code>visit_*</code> methods. What if we’re only interested
in <code>Identifier</code>? We don’t want to have to implement <code>visit_translation_unit</code> because we don’t care.</p>
<p>A simple fix to that is to give all those <code>visit_*</code> methods a default implementation… that does
nothing.</p>
<pre><code>trait Visitor {
  fn visit_identifier(&amp;mut self, _: &amp;mut Identifier) {}
  fn visit_translation_unit(&amp;mut self, _: &amp;mut TranslationUnit) {}
  // …
}
</code></pre>
<p>Let’s try it!</p>
<pre><code>struct ReplaceIdentifier&lt;'a&gt; {
  replacement: &amp;'a str
}

impl&lt;'a&gt; Visitor for ReplaceIdentifier&lt;'a&gt; {
  fn visit_identifier(&amp;mut self, ident: &amp;mut Identifier) {
    *ident = self.replacement.clone().into()
  }
}
</code></pre>
<p>And that’s all! We now have a visitor that can be used to traverse any AST and change any
<code>Identifier</code> to what we have set. For instance:</p>
<pre><code>let mut ast = …;
let mut visitor = ReplaceIdentifier { replacement: &quot;foobar&quot; }

// wait, how do we pass the visitor to the AST here?
</code></pre>
<p>Argh, we’re still missing something!</p>
<h2>Hosting visitors</h2>
<p>We need a <code>visit</code> function, like:</p>
<pre><code>fn visit&lt;V&gt;(ast: &amp;mut TranslationUnit, visitor: &amp;mut V) where V: Visitor
</code></pre>
<p>So let’s write it!</p>
<pre><code>fn visit&lt;V&gt;(ast: &amp;mut TranslationUnit, visitor: &amp;mut V) where V: Visitor {
  for external_decl in ast {
    // ah.
  }
}
</code></pre>
<p>We cannot call <code>visit</code> again on <code>ExternalDeclaration</code>. Seems like we need another trait! :D</p>
<pre><code>trait Host {
  fn visit&lt;V&gt;(&amp;mut Self, visitor: &amp;mut V) where V: Visitor;
}
</code></pre>
<p>Here, a type that implements <code>Host</code> means that it can call a <code>Visitor</code> on itself, and might pass it
down to children AST if any. Since we’re going to implement <code>Host</code> for all our AST types, we will be
able to do something like this:</p>
<pre><code>impl Host for TranslationUnit {
  fn visit&lt;V&gt;(&amp;mut Self, visitor: &amp;mut V) where V: Visitor {
    visitor.visit_translation_unit(self); // first, we have the visitor visit the AST node

    // then, for all children, we pass down the visitor!
    for external_decl in self {
      external_decl.visit(visitor);
    }
  }
}
</code></pre>
<p>And here we go. We are able to pass down the visitor, that will be called for each node. If you have
provided an implementation for a given <code>visit_*</code>, it will get invoked, otherwise, the default
implementation will fire – and it does nothing.</p>
<p>A simple optimization can be done, here. Since you might <em>know</em> that you’re done at a given level
regarding your visitor, we could add a way to make a <code>Host</code> stop visiting and go any deeper. For
this, we introduce a simple enum:</p>
<pre><code>enum Visit {
  Children, // keep visiting children
  Parent // stop visiting, go back to parent
}
</code></pre>
<p>And we change all the <code>Visitor</code> methods to return a <code>Visit</code>. That will give us the information we
need when implementing <code>Host::visit</code> now:</p>
<pre><code>impl Host for TranslationUnit {
  fn visit&lt;V&gt;(&amp;mut Self, visitor: &amp;mut V) where V: Visitor {
    let visit = visitor.visit_translation_unit(self); // first, we have the visitor visit the AST node

    if visit == Visit::Children {
      // then, for all children, we pass down the visitor!
      for external_decl in self {
        external_decl.visit(visitor);
      }
    }
  }
}
</code></pre>
<p>We also need to change the default implementation of the <code>visit_*</code> methods. My choice was to have
them return <code>Visit::Children</code> by default, because I think it’s a saner default – if people don’t
want to go any deeper in the AST, they will just have to dummy-implement the right method.</p>
<p>And we are done! The actual implementation is a bit more complex than that but is really really
close to what is described in this article. I’ll give you the example from the official
documentation so that can you can see how it’s <em>really</em> used:</p>
<pre><code>use glsl::syntax::{CompoundStatement, Expr, SingleDeclaration, Statement, TypeSpecifierNonArray};
use glsl::visitor::{Host, Visit, Visitor};
use std::iter::FromIterator;

let decl0 = Statement::declare_var(
  TypeSpecifierNonArray::Float,
  &quot;x&quot;,
  None,
  Some(Expr::from(3.14).into())
);

let decl1 = Statement::declare_var(
  TypeSpecifierNonArray::Int,
  &quot;y&quot;,
  None,
  None
);

let decl2 = Statement::declare_var(
  TypeSpecifierNonArray::Vec4,
  &quot;z&quot;,
  None,
  None
);

let mut compound = CompoundStatement::from_iter(vec![decl0, decl1, decl2]);

// our visitor that will count the number of variables it saw
struct Counter {
  var_nb: usize
}

impl Visitor for Counter {
  // we are only interested in single declaration with a name
  fn visit_single_declaration(&amp;mut self, declaration: &amp;mut SingleDeclaration) -&gt; Visit {
    if declaration.name.is_some() {
      self.var_nb += 1;
    }

    // do not go deeper
    Visit::Parent
  }
}

let mut counter = Counter { var_nb: 0 };
compound.visit(&amp;mut counter);
assert_eq!(counter.var_nb, 3);
</code></pre>
<h1>Future work</h1>
<p>The current state of <code>Visitor</code> is great but there is a drawback: your AST has to be mutable. You
might want to only traverse it read-only but have your visitor mutated. I might then modify slightly
the <code>Visitor</code> trait and <code>Host</code> one with <code>Host::visit_mut</code> if I think it’s needed.</p>
<p>Feel free to experiment around. Next work planned for <a href="https://crates.io/crates/glsl">glsl</a> – besides contributions – quasiquoting
variable interpolation.</p>
<p>Keep the vibes!</p>


guid: https://strongly-typed-thoughts.net/blog/luminance-0.3Tue, 06 Oct 2015 00:00:00 GMT
title: luminance-0.3 – Adding more texture kinds to the equation…
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance-0.3
feedurl: https://phaazon.net/blog/feed
content:
<h1>Unleashing the power of textures!</h1>
<p>From <a href="https://hackage.haskell.org/package/luminance-0.1">luminance-0.1</a> to
<a href="https://hackage.haskell.org/package/luminance-0.2">luminance-0.2</a> included, it was not possible to
use any texture types different than two-dimensional textures. This blog entry tags the new
release, <a href="https://hackage.haskell.org/package/luminance-0.3">luminance-0.3</a>, which adds support for
several kinds of texture.</p>
<h2>A bit more dimensions</h2>
<p><a href="https://hackage.haskell.org/package/luminance-0.3/docs/Graphics-Luminance-Texture.html#t:Texture1D"><code>Texture1D</code></a>,
<a href="https://hackage.haskell.org/package/luminance-0.3/docs/Graphics-Luminance-Texture.html#t:Texture2D"><code>Texture2D</code></a> and
<a href="https://hackage.haskell.org/package/luminance-0.3/docs/Graphics-Luminance-Texture.html#t:Texture3D"><code>Texture3D</code></a>
are all part of the new release. The interface has changed – hence the breaking changes yield a
major version increment – and I’ll explain how it has.</p>
<p>Basically, textures are now fully polymorphic and are constrained by a typeclass:
<a href="https://hackage.haskell.org/package/luminance-0.3/docs/Graphics-Luminance-Texture.html#t:Texture"><code>Texture</code></a>.
That typeclass enables ad hoc polymorphism. It is then possible to add more texture types without
having to change the interface, which is cool. Like everything else in luminance, you just have to
ask the typesystem which kind of texture you want, and everything will be taken care of for you.</p>
<p>Basically, you have three functions to know:</p>
<ul>
<li><a href="https://hackage.haskell.org/package/luminance-0.3/docs/Graphics-Luminance-Texture.html#v:createTexture"><code>createTexture</code></a>,
which is used to create a new texture ;</li>
<li><a href="https://hackage.haskell.org/package/luminance-0.3/docs/Graphics-Luminance-Texture.html#v:uploadSub"><code>uploadSub</code></a>,
used to upload texels to a subpart of the texture ;</li>
<li><a href="https://hackage.haskell.org/package/luminance-0.3/docs/Graphics-Luminance-Texture.html#v:fillSub"><code>fillSub</code></a>,
used to fill – <em>clear</em> – a subpart of the texture with a given value.</li>
</ul>
<p>All those functions work on <code>(Texture t) =&gt; t</code>, so it will work with all kinds of texture.</p>
<h2>Cubemaps</h2>
<p><a href="https://hackage.haskell.org/package/luminance-0.3/docs/Graphics-Luminance-Texture.html#t:Cubemap"><code>Cubemap</code>s</a>
are also included. They work like other textures but add the concept of <em>faces</em>. Feel free to dig in
the documentation for further details.</p>
<h1>What’s next?</h1>
<p>I need to find a way to wrap <em>texture arrays</em>, which are very nice and useful for
<em>layered rendering</em>. After that, I’ll try to expose the change to the framebuffers so that we can
create framebuffers with cubemaps or that kind of cool feature.</p>
<p>In the waiting, have a good a week!</p>


title: Contravariance and luminance to add safety to uniforms
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance_uniform_contravariance
feedurl: https://phaazon.net/blog/feed
guid:
https://strongly-typed-thoughts.net/blog/luminance_uniform_contravarianceSun, 23 Aug 2015 00:00:00 GMT

content:
<p>It’s been a few days I haven’t posted about <a href="https://github.com/phaazon/luminance">luminance</a>. I’m
on holidays, thus I can’t be as involved in the development of the graphics framework as I’m used to
on a daily basis. Although I’ve been producing less in the past few days, I’ve been actively
thinking about something very important: <a href="https://www.opengl.org/wiki/Uniform_%28GLSL%29">uniform</a>.</p>
<h1>What people usually do</h1>
<p>Uniforms are a way to pass data to shaders. I won’t talk about <em>uniform blocks</em> nor <em>uniform
buffers</em> – I’ll make a dedicated post for that purpose. The common OpenGL uniform flow is the
following:</p>
<ol>
<li>you ask OpenGL to retrieve the location of a GLSL uniform through the function
<code>glGetUniformLocation</code>, or you can use an explicit location if you want to handle the semantics
on your own ;</li>
<li>you use that location, the identifier of your shader program and send the actual values with
the proper <code>glProgramUniform</code>.</li>
</ol>
<p>You typically don’t retrieve the location each time you need to send values to the GPU – you only
retrieve them once, while initializing.</p>
<p>The first thing to make uniforms more elegant and safer is to provide a typeclass to provide a
shared interface. Instead of using several functions for each type of uniform – <code>glProgramUniform1i</code>
for <code>Int32</code>, <code>glProgramUniform1f</code> for <code>Float</code> and so on – we can just provide a function that will
call the right OpenGL function for the type:</p>
<pre><code class="language-haskell">class Uniform a where
  sendUniform :: GLuint -&gt; GLint -&gt; a -&gt; IO ()

instance Uniform Int32 where
  sendUniform = glProgramUniform1i

instance Uniform Float where
  sendUniform = glProgramUniform1f

-- and so on…
</code></pre>
<p>That’s the first step, and I think everyone should do that. However, that way of doing has several
drawbacks:</p>
<ul>
<li>it still relies on side-effects; that is, we can call <code>sendUniform</code> pretty much everywhere ;</li>
<li>imagine we have a shader program that <strong>requires</strong> several uniforms to be passed each time we
draw something; what happens if we forget to call a <code>sendUniform</code>? If we haven’t sent the
uniform yet, we might have an undefined behavior. If we already have, we will <em>override</em> all
future draws with that value, which is very wrong… ;</li>
<li>with that way of representing uniforms, we have a very imperative interface; we can have a more
composable and pure approach than that, hence enabling us to gain in power and flexibility.</li>
</ul>
<h1>What luminance used to do</h1>
<p>In my <a href="https://github.com/phaazon/luminance">luminance</a> package, I used to represent uniforms as
values.</p>
<pre><code class="language-haskell">newtype U a = U { runU :: a -&gt; IO () }
</code></pre>
<p>We can then alter the <code>Uniform</code> typeclass to make it simpler:</p>
<pre><code class="language-haskell">class Uniform a where
  toU :: GLuint -&gt; GLint -&gt; U a

instance Uniform Int32 where
  toU prog l = U $ glProgramUniform1i prog l

instance Uniform Float where
  toU prog l = U $ glProgramUniform1f prog l
</code></pre>
<p>We also have a pure interface now. I used to provide another type, <code>Uniformed</code>, to be able to
<em>send</em> uniforms without exposing <code>IO</code>, and an operator to accumulate uniforms settings, <code>(@=)</code>:</p>
<pre><code class="language-haskell">newtype Uniformed a = Uniformed { runUniformed :: IO a } deriving (Applicative,Functor,Monad)

(@=) :: U a -&gt; a -&gt; Uniformed ()
U f @= a = Uniformed $ f a
</code></pre>
<p>Pretty simple.</p>
<h1>The new uniform interface</h1>
<p>The problem with that is that we still have the completion problem and the side-effects, because we
just wrap them without adding anything special – <code>Uniformed</code> is isomorphic to <code>IO</code>. We have no way
to create a type and ensure that <em>all</em> uniforms have been sent down to the GPU…</p>
<h2>Contravariance to save us!</h2>
<p>If you’re an advanced <strong>Haskell</strong> programmer, you might have noticed something very interesting
about our <code>U</code> type. It’s contravariant in its argument. What’s cool about that is that we could then
create new uniform types – new <code>U</code> – by contramapping over those types! That means we can enrich
the scope of the hardcoded <code>Uniform</code> instances, because the single way we have to get a <code>U</code> is
to use <code>Uniform.toU</code>. With contravariance, we can – in theory – extend those types to <strong>all types</strong>.</p>
<p>Sounds handy eh? First thing first, contravariant functor. A contravariant functor is a functor that
flips the direction of the morphism:</p>
<pre><code class="language-haskell">class Contravariant f where
  contramap :: (a -&gt; b) -&gt; f b -&gt; f a
  (&gt;$) :: b -&gt; f b -&gt; f a
</code></pre>
<p><code>contramap</code> is the <em>contravariant</em> version of <code>fmap</code> and <code>(&gt;$)</code> is the contravariant version of
<code>(&lt;$)</code>. If you’re not used to contravariance or if it’s the first time you see such a type
signature, it might seem confusing or even <strong>magic</strong>. Well, that’s the mathematic magic in the
place! But you’ll see just below that there’s no magic no trick in the implementation.</p>
<p>Because <code>U</code> is contravariant in its argument, we can define a <code>Contravariant</code> instance:</p>
<pre><code class="language-haskell">instance Contravariant U where
  contramap f u = U $ runU u . f
</code></pre>
<p>As you can see, nothing tricky here. We just apply the <code>(a -&gt; b)</code> function on the input of the
resulting <code>U a</code> so that we can pass it to <code>u</code>, and we just <code>runU</code> the whole thing.</p>
<p>A few friends of mine – not <strong>Haskeller</strong> though – told me things like <em>“That’s just theory
bullshit, no one needs to know what a contravariant thingy stuff is!”</em>. Well, here’s an example:</p>
<pre><code class="language-haskell">newtype Color = Color {
    colorName :: String
  , colorValue :: (Float,Float,Float,Float)
  }
</code></pre>
<p>Even though we have an instance of <code>Uniform</code> for <code>(Float,Float,Float,Float)</code>, there will never be an
instance of <code>Uniform</code> for <code>Color</code>, so we can’t have a <code>U Color</code>… Or can we?</p>
<pre><code class="language-haskell">uColor = contramap colorValue float4U
</code></pre>
<p>The type of <code>uColor</code> is… <code>U Color</code>! That works because contravariance enabled us to <em>adapt</em> the
<code>Color</code> structure so that we end up on <code>(Float,Float,Float,Float)</code>. The contravariance property is
then a very great ally in such situations!</p>
<h2>More contravariance</h2>
<p>We can even dig in deeper! Something cool would be to do the same thing, but for several fields.
Imagine a mouse:</p>
<pre><code class="language-haskell">data Mouse = Mouse {
    mouseX :: Float
  , mouseY :: Float
  }
</code></pre>
<p>We’d like to find a cool way to have <code>U Mouse</code>, so that we can send the mouse cursor to shaders.
We’d like to contramap over <code>mouseX</code> and <code>mouseY</code>. A bit like with <code>Functor</code> + <code>Applicative</code>:</p>
<pre><code class="language-haskell">getMouseX :: IO Float
getMouseY :: IO Float

getMouse :: IO Mouse
getMouse = Mouse &lt;$&gt; getMouseX &lt;*&gt; getMouseY
</code></pre>
<p>We could have the same thing for contravariance… And guess what. That exists, and that’s called
<strong>divisible contravariant functors</strong>! A <code>Divisible</code> contravariant functor is the exact contravariant
version of <code>Applicative</code>!</p>
<pre><code class="language-haskell">class (Contravariant f) =&gt; Divisible f where
  divide :: (a -&gt; (b,c)) -&gt; f b -&gt; f c -&gt; f a
  conquer :: f a
</code></pre>
<p><code>divide</code> is the contravariant version of <code>(&lt;*&gt;)</code> and <code>conquer</code> is the contravariant version of
<code>pure</code>. You know that <code>pure</code>’s type is <code>a -&gt; f a</code>, which is isomorphic to <code>(() -&gt; a) -&gt; f a</code>. Take
the contravariant version of <code>(() -&gt; a) -&gt; f a</code>, you end up with <code>(a -&gt; ()) -&gt; f a</code>. <code>(a -&gt; ())</code> is
isomorphic to <code>()</code>, so we can simplify the whole thing to <code>f a</code>. Here you have <code>conquer</code>. <em>Thank you
to Edward Kmett for helping me understand that!</em></p>
<p>Let’s see how we can implement <code>Divisible</code> for <code>U</code>!</p>
<pre><code class="language-haskell">instance Divisible U where
  divide f p q = U $ \a -&gt; do
    let (b,c) = f a
    runU p b
    runU q c
  conquer = U . const $ pure ()
</code></pre>
<p>And now let’s use it to get a <code>U Mouse</code>!</p>
<pre><code class="language-haskell">let uMouse = divide (\(Mouse mx my) -&gt; (mx,my)) mouseXU mouseYU
</code></pre>
<p>And here we have <code>uMouse :: U Mouse</code>! As you can see, if you have several uniforms – for each fields
of the type, you can <code>divide</code> your type and map all fields to the uniforms by applying several times
<code>divide</code>.</p>
<p>The current implementation is almost the one shown here. There’s also a <code>Decidable</code> instance, but
I won’t talk about that for now.</p>
<p>The cool thing about that is that I can lose the <code>Uniformed</code> monadic type and rely only on <code>U</code>.
Thanks to the <code>Divisible</code> typeclass, we have completion, and we can’t override future uniforms then!</p>
<hr />
<p>I hope you’ve learnt something cool and useful through this. Keep in mind that category abstractions 
<strong>are powerful</strong> and are useful in some contexts.</p>
<p>Keep hacking around, keep being curious. A <strong>Haskeller</strong> never stops learning! And that’s what so
cool about <strong>Haskell</strong>! Keep the vibe, and see you another luminance post soon!</p>


guid: https://strongly-typed-thoughts.net/blog/rust-2020-ideasThu, 31 Oct 2019 15:15:00 GMT
title: My Rust 2020 ideas
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/rust-2020-ideas
feedurl: https://phaazon.net/blog/feed
content:
<p>This blog article is a small reply to the
<a href="https://blog.rust-lang.org/2019/10/29/A-call-for-blogs-2020.html">public call for blog posts 2020 in the Rust community</a>.
I will express what I would like Rust to go to, keeping in mind that it’s solely my ideas and
opinions others’ might differ.</p>
<p>The points expressed here are written by decreasing priority, starting with the feature I would
like the most to see implemented as soon as possible.</p>
<!-- vim-markdown-toc GFM -->
<ul>
<li><a href="#rank-n-quantification">Rank-N quantification</a></li>
<li><a href="#kinds">Kinds</a></li>
<li><a href="#gat">GAT</a></li>
<li><a href="#polymorphic-literals">Polymorphic literals</a></li>
<li><a href="#custom-operators">Custom operators</a></li>
<li><a href="#cargo-dependencies">Cargo dependencies</a></li>
<li><a href="#sealed-traits--private-trait-items">Sealed traits / private trait items</a></li>
<li><a href="#feature-discoverability">Feature discoverability</a></li>
<li><a href="#conclusion">Conclusion</a></li>
</ul>
<!-- vim-markdown-toc -->
<h1>Rank-N quantification</h1>
<p>It’s for sure one among the two features I miss the most from Haskell. Rank-N quantification is a
feature that, when I discovered it almost 9 years ago, changed a lot of things in my way of thinking
and desiging interfaces.</p>
<p>For those not used to it or those having no idea what rank-N quantification is, let me explain
with simple words by taking an example.</p>
<p>Imagine a function that works on a vector of <code>u32</code>:</p>
<pre><code class="language-rust">fn process(v: Vec&lt;u32&gt;)
</code></pre>
<p>That function is <em>monomorphic</em>. You cannot, at compile-time, get several flavours of <code>process</code>.
But now imagine you want to process lots of things without actually requiring the element type
to be <code>u32</code> but, let’s say, <code>T: Into&lt;u32&gt;</code>:</p>
<pre><code class="language-rust">fn process&lt;T&gt;(v: Vec&lt;T&gt;) where T: Into&lt;u32&gt;
</code></pre>
<p>That function has a type variable, <code>T</code>, and we say that is has a <em>rank of 1</em>, or it’s a <em>rank-1</em>
function. If you add more variables, they all remain at the same <em>level</em>, so the function is
still <em>rank-1</em>:</p>
<pre><code class="language-rust">fn process&lt;T, Q&gt;(v: Vec&lt;T&gt;, other: Option&lt;Q&gt;) where T: Into&lt;u32&gt;, Q: Display
</code></pre>
<p>Now, imagine a function that would take as sole argument a function which takes a <code>u32</code> and
returns a <code>String</code>, for instance:</p>
<pre><code class="language-rust">fn foo&lt;F&gt;(f: F) where F: Fn(u32) -&gt; String {
  println!(&quot;the function returned {}&quot;, f(123));
}
</code></pre>
<p>That function is still <em>rank-1</em>. But now, imagine that we would like to express the idea that the
function that is passed as argument <em>must</em> work for <em>any</em> <code>T: Debug</code> instead of <code>u32</code>. Here, the
<em>any</em> word is important, because it means that you must pass a polymorphic function to <code>foo</code>, which
will get monomorphized <em>inside</em> the body of <code>foo</code>. If you’ve thought about this:</p>
<pre><code class="language-rust">fn foo&lt;F, T&gt;(f: F) where F: Fn(T) -&gt; String, T: Debug {
  println!(&quot;the function returned {}&quot;, f(123u32));
}
</code></pre>
<p>Then you’re wrong, because that function cannot compile. The reason for this is that the type of <code>f</code>
is <code>F: Fn(T) -&gt; String</code> and you try to pass <code>123u32</code> instead of <code>T</code>. That function definition cannot
work because the body would force <code>T</code> to be <code>u32</code>. The problem here is that, currently, Rust doesn’t
allow us to do what we want to: <code>T</code> shouldn’t be monomorphized <em>at the same rank as</em> <code>F</code>, because
<code>T</code> will be chosen by the implementation of <code>foo</code>, not the caller!</p>
<p>I would like this:</p>
<pre><code class="language-rust">fn foo&lt;F&gt;(f: F) where F: for&lt;T&gt; Fn(T) -&gt; String where T: Debug;

// or
fn foo&lt;F&gt;(f: F) where F: for&lt;T: Debug&gt; Fn(T) -&gt; String;
</code></pre>
<p>We call that a <em>rank-2</em> function, because it has two <em>levels</em> / <em>ranks</em> of type variables. We could
use it this way:</p>
<pre><code class="language-rust">fn foo&lt;F&gt;(f: F) where F: for&lt;T&gt; Fn(T) -&gt; String where T: Debug {
  println!(&quot;the function returned {}&quot;, f(123u32);
  println!(&quot;the function returned {}&quot;, f(&quot;Hello, world!&quot;);
}
</code></pre>
<p>You can imagine rank-N quantification by nesting HRTB syntax:</p>
<pre><code class="language-rust">// a rank-3 function
fn foo&lt;F&gt;(f: F) where F: for&lt;T&gt; Fn() -&gt; T where T: for&lt;X&gt; Into&lt;X&gt; where X: Debug;
</code></pre>
<p>But it’s rarely needed and I struggle to find a real usecase for them (but there are!). From my
Haskell experience, we really really rarely need more than rank-2 quantification.</p>
<p>You can find more in-details thoughts of that feature on a previous article of mine,
<a href="https://phaazon.net/blog/rank-n-rust">here</a>.</p>
<h1>Kinds</h1>
<p>“Kinds” is the second feature I would love to see in Rust. For those who don’t know what they are,
consider:</p>
<ul>
<li>A <em>value</em> is something that lives at runtime. In Rust, it has a <em>type</em>. You can refer to values
directly with literals or you can bind them via <em>let bindings</em>, for instance. In <code>let x = 3</code>,
<code>3</code> is a value and <code>x</code> is a binding to that value.</li>
<li>A <em>type</em> is like a <em>value</em> that lives at compile-time. You can refer to types directly or via
<em>type variables</em>. For instance, <code>u32</code> is a type and in <code>fn foo&lt;T&gt;()</code>, <code>T</code> is a type variable.
It’s actually a <em>free variable</em>, here. Currently, Rust stops here.</li>
</ul>
<p>In Haskell but also Idris, ATS, Coq and many others, types have types too. We name those kinds.
To understand what it means:</p>
<ul>
<li>A type is just a <em>label</em> on a set. <code>u32</code> is a label (really, imagine <code>&quot;u32&quot;</code>) of a very big
set that contains <strong>all the possible</strong> values that can be labelled as <code>u32</code>. You find in that
set <code>0</code>, <code>1</code>, <code>34</code>, <code>2390743</code>, etc.</li>
<li>A kind is just a <em>label</em> on a set, too. But that set doesn’t contain values; it contains types.
A kind is just a labelled set of types.</li>
</ul>
<p>For instance, imagine the kind <code>Number</code>. You can put in that set the types <code>u32</code>, <code>i32</code>, <code>usize</code>,
<code>f32</code>, etc. But now imagine: type variables are to types what variables are to values. What would
be a <em>kind variable</em>? Well, it would be something that would allow us to give more details about
what a type should be. For instance:</p>
<pre><code class="language-rust">trait Functor {
  fn map&lt;A, B, F&gt;(self: Self&lt;A&gt;, f: F) -&gt; Self&lt;B&gt;;
}

impl Functor for Option {
  fn map&lt;A, B, F&gt;(self: Self&lt;A&gt;, f: F) -&gt; Self&lt;B&gt; {
    self.map(f)
  }
}

// whatever the type of functor, just ignore its content and replace with ()
// we could also introduce a type variable A and use fct: Fct&lt;A&gt; but since we don’t
// need it, we use _
//
// The &lt;Fct&lt;_&gt;&gt; marks the kind of Fct (i.e. its kind is Type -&gt; Type, as in it expects
// a type to be type)
fn void&lt;Fct&lt;_&gt;&gt;(fct: Fct&lt;_&gt;) -&gt; Fct&lt;()&gt; where Fct: Functor {
  fct.map(|_| ())
}
</code></pre>
<p>Currently, that syntax doesn’t exist and I don’t even know how it would be formalized. The <code>void</code>
function above looks okay to me but not the <code>trait</code> definition. The syntax <code>T&lt;_, _&gt;</code> would
declare a type which must has two type variables, etc.</p>
<h1>GAT</h1>
<p><a href="https://github.com/rust-lang/rust/issues/44265">GATs</a> are a bit akin to kinds in the sense that they
allow to express type constructors (i.e. which kinds are, for instance, <code>Type -&gt; Type</code>, if they only
have one type variable).</p>
<p>That’s a feature I need a lot in several crates of mine, so I hope it will be implemented and stable
soon! :)</p>
<h1>Polymorphic literals</h1>
<p>Something I want a lot too and hasn’t been discussed a lot (I might write an RFC for that because
I want it very badly). What it means is that:</p>
<pre><code class="language-rust">let x = 3;
</code></pre>
<p>The type of <code>x</code> here would be polymorphic as <code>T: FromLit&lt;usize&gt;</code>. We would have several implementors
and it would go like this:</p>
<pre><code class="language-rust">pub trait FromLit&lt;L&gt;: L: Lit {
  const fn from_lit(lit: L) -&gt; Self;
}

// blanket impl
impl&lt;L&gt; FromLit&lt;L&gt; where L: Lit {
  const fn from_lit(lit: L) -&gt; Self {
    lit
  }
}

// generated by rustc
impl Lit for usize {}
impl Lit for isize {}
impl Lit for u32 {}
impl Lit for &amp;'static str {}
// …
</code></pre>
<p>This would allow us to do something like that:</p>
<pre><code class="language-rust">pub enum Expr {
  ConstBool(bool),
  ConstI32(i32),
  // …
}

impl FromLit&lt;bool&gt; for Expr {
  const fn from_lit(lit: L) -&gt; Self {
    Expr::ConstBool(lit)
  }
}

// in a function
let expr: Expr = false;
</code></pre>
<p>As a rationale, Haskell has that in its base language since forever and under the language extension
called <code>OverloadedStrings</code> and <code>OverloadedLists</code> for strings and lists.</p>
<h1>Custom operators</h1>
<p>A feature that wouldn’t make everyone happy, so I’m pretty sure it will not be in the Rust 2020 roadmap
(and maybe never end up in Rust at all, sadly), but I think it’s worth mentioning it.</p>
<p>I would be able to create custom operators. The reason for this is simple:
<a href="https://wiki.haskell.org/Embedded_domain_specific_language">EDSLs</a>. I love EDSLs. Having the possibility
to enrich expressiveness via custom operators is something I’ve been wanting for quite a while now and
I’m so surprised people haven’t arised that concern yet.</p>
<p>I know there is concerns from people who know the
<a href="http://hackage.haskell.org/package/lens">Haskell lens library</a> and its infamous lists of horrible and
awful operators, but that’s not a reason to block such a feature to me, for two reasons:</p>
<ul>
<li><code>lens</code> is really extreme is likely the sole real problem in the Haskell ecosystem.</li>
<li>We could mitigate that <em>fear</em> by forcing custom operators to have a <em>function</em> associated with the
operator, so that people who don’t want to use the custom operator can still use a correctly named
function.</li>
</ul>
<p>I’m a huge partisan of the idea that there are billions of people speaking Chinese, a language very
cryptic <em>to me</em>, because I just cannot speak Chinese. Yet it doesn’t prevent billions of people speaking
Chinese on a daily basis without any problem. It’s always a question of learning and an operator should
be perceived as a function name. Stop spreading fear about readability: a convoluted function name is
also pretty hard to read.</p>
<p>To mitigate fear even further, there are several very good operators in Haskell that are actually very very
simple to understand and memorize:</p>
<ul>
<li>The <code>fmap</code> function operator version is <code>&lt;$&gt;</code>.</li>
<li>When applying <code>fmap</code> to ignore what’s inside a functor and replacing it with a constant, we use
<code>fmap (const 32)</code>, for instance — or <code>fmap (\_ -&gt; 32)</code> for the lambda version. You can use <code>&lt;$&gt;</code> too,
<code>const 32 &lt;$&gt; [1, 2, 3]</code>. But there’s a very logical and simple operator to remember. Notice how the
constant value is at the left of the <code>&lt;$&gt;</code> operator? Then, you can do the exact same thing with
<code>32 &lt;$ [1, 2, 3]</code>. On the same level, if you prefer to put the value on the right side:
<code>[1, 2, 3] $&gt; 32</code>. Simple.</li>
<li>The <code>&lt;|&gt;</code> operator. It has that <code>|</code> in it, which is often <code>OR</code> in most languages. And guess what: that
operator is the <em>alternative operator</em>. <code>a &lt;|&gt; b</code> equals <code>a</code> if it’s <em>true</em> or <code>b</code> if not. The definition
of truth is up to the type, but for <code>Maybe a</code> — <code>Option&lt;T&gt;</code> in Rust, <em>true</em> is <code>Just _</code> (<code>Some(_)</code>) and
<em>false</em> is <code>Nothing</code> (<code>None</code>). See how easy it is?</li>
<li>In <a href="http://hackage.haskell.org/package/parsec">parsec</a>, one of the most famous Haskell parser, there’s
an operator you can use to provide more descriptive error messages for <em>all parsers</em> (even sub-parsers!):
the <code>&lt;?&gt;</code> operator. Once again, the <code>?</code> reminds a question, information, etc.</li>
<li>The <code>.</code> operator in Haskell composes two functions. It looks closely to the <code>∘</code> math notation.</li>
<li>The monadic bind is <code>&gt;&gt;=</code>, which expects the function on the right hand side… Guess what the
<code>=&lt;&lt;</code> operator does. The exact same thing, but expects the function to be on the left hand side.</li>
<li>When dealing with paths, the <code>&lt;/&gt;</code> operator allows to separate path parts without redundant <code>/</code>. The
<code>&lt;.&gt;</code> operator allows you to write the extension name!
<code>&quot;/home/phaazon&quot; &lt;/&gt; spareProjectsDir &lt;/&gt; &quot;rust/rfcs&quot; &lt;/&gt; lastRFC &lt;.&gt; &quot;md&quot;</code>.</li>
<li>Etc. etc.</li>
</ul>
<h1>Cargo dependencies</h1>
<p>A huge topic, but basically, I hope that <code>cargo</code> can now resolve dependencies without accepting
several versions of a crate, but instead resolves them by traversing the whole dependency graph.</p>
<p>This is often needed on my side as I like to make a crate compatible with several dependency
versions, so that people who don’t update often can still have their clients benefits from updates
on my side. It’s expressed as SemVer ranges (e.g. <code>stuff = &quot;&gt;=0.4, &lt; 0.9&quot;</code>) but <code>cargo</code> will take
the <em>best</em> one it knows. Basically, if you have such a dependency in project <code>A</code> and you depend on
<code>B</code> which has <code>stuff = &quot;0.5&quot;</code>, then you will end up with both <code>stuff-0.5</code> and <code>stuff-0.8</code> in your
graph dependency, which to me is very wrong. Intersecting dependencies <em>should</em> only bring
<code>stuff-0.5</code>, because it’s the highest minimal version that satisfies every crates depending on it
in the dependency graph.</p>
<h1>Sealed traits / private trait items</h1>
<p>I talked about it <a href="https://phaazon.net/blog/rust-traits-privacy">here</a>, but basically, I want to be
able to annotate trait’s items with visibility qualifiers (i.e. <code>pub</code>, <code>pub(crate)</code>, etc.) so that
I can implement a trait in a crate without having people depending on my crate see the guts of the
trait.</p>
<p>Sealed traits prevent people from implementing the trait (first new concept) and private trait
items both prevent people from implementing the trait but they also prevent them from seeing what’s
inside.</p>
<h1>Feature discoverability</h1>
<p>Long discussion occurring <a href="https://phaazon.net/blog/rust-features-documentation">here</a>. Basically,
since <em>features</em> are parts of the language (both via <code>cfg</code> attributes and in <code>Cargo.toml</code>), it
would neat to be able to show them in <code>rustdoc</code> to help people discover what’s possible to do with
a crate, instead of opening the <code>Cargo.toml</code> on GitHub / whatever you’re using.</p>
<h1>Conclusion</h1>
<p>So that’s all for me and what I would love Rust to go towards. I have no idea whether people from
the Rust project will actually read 10% of what I just wrote; I feel like I just made a wish list
for Christmas.</p>
<p>Thank you for having read. Thank you for contributing to Rust and making it the best language
in the world. And as always, keep the vibes and <strong>please</strong> let’s talk on Reddit! :)</p>


guid: https://strongly-typed-thoughts.net/blog/resource_managementTue, 16 Jun 2015 00:00:00 GMT
title: Asset management in a real time 3D engine in Haskell
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/resource_management
feedurl: https://phaazon.net/blog/feed
content:
<h1>Foreword</h1>
<p>In a real time rendering system, it’s not uncommon finding constructs about
<em>assets</em>. One famous construct is the <em>resource manager</em>. A <em>resource manager</em> is
responsible of several tasks, among:</p>
<ul>
<li>providing a simple interface to load objects from disk <em>(1)</em> ;</li>
<li>ensuring we don’t try to load the same object twice <em>(2)</em> ;</li>
<li>resolving dependencies between resources <em>(3)</em>.</li>
</ul>
<p>The first point is obvious, but the two others are less intuitive. <em>(2)</em> is
important when the user <em>might</em> try to load the same object several times – for
instance, a car model, or a character or a weapon. The most known strategy to
prevent such a situation from happening is by using a <em>software cache</em>.</p>
<p>A <em>software cache</em> – let’s just say <em>cache</em> – is an opaque object that loads the
object on the first request, then just returns that object for future same
requests. For instance, consider the following requests and the corresponding
<em>cache</em> behavior:</p>
<ol>
<li>load &quot;wood.png&quot; -&gt; not cached ; loading ; return</li>
<li>load &quot;grass.png&quot; -&gt; not cached ; loading ; return</li>
<li>load &quot;wood.png&quot; -&gt; cached ; return</li>
<li>load &quot;metal.png&quot; -&gt; not cached ; loading ; return</li>
<li>load &quot;metal.png&quot; -&gt; cached ; return</li>
<li>etc.</li>
</ol>
<p>That behavior is very nice because it will spare a lot of computations and
memory space.</p>
<p><em>(3)</em> is about dependencies. For instance, when you load a car model, you might
need to load its textures as well. Well, not really <em>load</em>. Consider the
following:</p>
<ol>
<li>load &quot;car.mesh&quot; -&gt; not cached
1. load &quot;metal_car.png&quot; -&gt; not cached ; loading ; return
2. loading ; return</li>
<li>load &quot;metal_car.png&quot; -&gt; cached ; return</li>
<li>load &quot;other_car.mesh&quot; -&gt; not cached
1. load &quot;metal_car.png&quot; -&gt; cached ; return
2. return</li>
<li>load &quot;car.mesh&quot; -&gt; cached ; return</li>
</ol>
<p>You got the idea. <em>(3)</em> needs <em>(2)</em> to be efficient.</p>
<h2>Possible implementations</h2>
<h3>Singleton</h3>
<p>In imperative languages and especially in those that support template and/or
generics, people tend to implement the cache system with an ugly <em>design
pattern</em> – which is actually an <em>anti design pattern</em> : <em>singleton</em>. Each type
of resource is assigned a manager by using a template parameter, and then if
a manager needs to load a dependency, it just has to reference the corresponding
manager by stating the type in the template parameter :</p>
<pre><code>Model &amp; getResource&lt;Model&gt;(std::string const &amp;name) {
  Texture &amp;dependency = getResource&lt;Texture&gt;(...);
  ...
}
</code></pre>
<p>That way of doing might sound great, but eh, singletons are just global variables
with a unicity constraint. We don’t want that.</p>
<h3>Explicit pure store</h3>
<p>We can use an explicit store object. That is, some kind of <em>map</em>. For instance,
the store that holds <em>textures</em> would have a type like (in Haskell):</p>
<pre><code>textureStore :: Map String Texture
</code></pre>
<p>A model store would have the following type:</p>
<pre><code>modelStore :: Map String Model
</code></pre>
<p>And each stores is assigned a function; <code>loadTexture</code>, <code>loadModel</code>, and so on.</p>
<p>There are several drawbacks if we go that way. First, we have to carry all stores
when using their functions. Some functions might need other stuff in order to
resolve dependencies. Secondly, because of explicit state, we need to manually
accumulate state! A loading function would have such a following type:</p>
<pre><code>loadTexture :: Map String Texture -&gt; String -&gt; m (Texture,Map String Texture)
</code></pre>
<p>That will expose <strong>a lot</strong> of boilerplate to the user, and we don’t want that.</p>
<h3>Implicit pure store</h3>
<p>We can enhance the explicit store by putting it into some kind of context; for
instance, in <code>MonadState</code>. We can then write <code>loadTexture</code> to make it nicer to
use:</p>
<pre><code>loadTexture :: (MonadState (Map String Texture) m,...)
            =&gt; String
            -&gt; m Texture
</code></pre>
<p>There is a problem with that. What happens when we add more types? For instance if
we want to handle textures <strong>and</strong> models? <code>MonadState</code> has a <em>type family
constraint</em> that forbids two instances for the pair <code>s m</code>. The following is not
allowed and will raise a compiler error:</p>
<pre><code>instance MonadState (Map String Texture) MyState where
  ...

instance MonadState (Map String Model) MyState where
  ...
</code></pre>
<p>The solution to that problem is to have the carried state a polymorphic type and
use typeclass constraint to extract and modify the map we want:</p>
<pre><code>class HasMap a s where
  extractMap :: s -&gt; Map String a
  modifyMap :: (Map String a -&gt; Map String a) -&gt; s -&gt; s
</code></pre>
<p>With that, we can do something like this:</p>
<pre><code>loadTexture :: (MonadState s m,HasMap Texture s,...)
            =&gt; String
            -&gt; m Texture

loadModel :: (MonadState s m,HasMap Texture s,HasMap Model s,...)
          =&gt; String
          -&gt; m Model
</code></pre>
<p>However, we hit a new issue here. What are <code>s</code> and <code>m</code>? Well, <code>m</code> doesn’t really
matter. For simplicity, let’s state we’re using a <em>monad transformer</em>; that is,
we use <code>StateT s m</code> as monad.</p>
<p>We still need <code>s</code>. The problem is that <code>s</code> has to be provided by the user. Worse,
they have to implement <strong>all instances</strong> we need so that the loading functions may
work. Less boilerplate than the explicit store solution, but still a lot of
boilerplate. Imagine you provide a type for <code>s</code>, like <code>Cache</code>. Expending the
cache to support new types – like user-defined ones – will be more extra
boilerplate to write.</p>
<h3>Closures</h3>
<p>The solution I use in my engine might not be the perfect solution. It’s not
<a href="https://en.wikipedia.org/wiki/Referential_transparency_(computer_science)">referentially transparent</a>,
an important concept in Haskell. However, Haskell is not designed to be used in
convenient situations only. We’re hitting a problematic situation. We need to make
a compromise between elegance and simplicity.</p>
<p>The solution required the use of <em>closures</em>. If you don’t know what a closure is,
you should check out the <a href="https://en.wikipedia.org/wiki/Closure_(computer_programming)">wikipedia page</a>
for a first shot.</p>
<p>The idea is that our loading functions will perform some <code>IO</code> operations to
load objects. Why not putting the cache <em>directly</em> in that function? We’d have a
function with an opaque and invisible associated state. Consider the following:</p>
<pre><code>type ResourceMap a = Map String a

getTextureManager :: (MonadIO m,...)
                  =&gt; m (String -&gt; m Texture)
getTextureManager = do
  ref &lt;- newIORef empty
  pure $ \name -&gt; do
    -- we can use the ref ResourceMap to insert / lookup value in the map
    -- thanks to the closure!
</code></pre>
<p>That solution is great because now, a manager is just a function. How would you
implement <code>getModelManager</code>? Well:</p>
<pre><code>getModelManager :: (MonadIO m,...)
                =&gt; (String -&gt; m Texture)
                -&gt; m (String -&gt; m Model)
getModelManager loadTexture = ...
</code></pre>
<p>We can then get the loader functions with the following:</p>
<pre><code>loadTexture &lt;- getTextureManager
loadModel &lt;- getModelManager loadTexture
</code></pre>
<p>And you just have to pass those functions around. The cool thing is that you can
wrap them in a type in your library and provide a function that initializes them
all at once – I do that in my engine. Later on, the user can extend the available
managers by providing new functions for new types. In my engine, I provide a few
functions like <code>mkResourceManager</code> that hides the <code>ResourceMap</code>, providing two
functions – one for lookup in the map, one for inserting into the map.</p>
<h1>Conclusion</h1>
<p>I truly believe that my solution is a good compromise between elegance and ease.
It has a lot of advantages:</p>
<ul>
<li>simple to use ;</li>
<li>simple to implement; you just have to play around with closures ;</li>
<li>dependencies resolving is easy to add and hidden once the functions are 
generated ;</li>
<li>little runtime overhead (due to closures, might be optimized away by the
compiler though) ;</li>
<li>can be easily extended by the user to support custom/new types ;</li>
<li>if correctly used, implementations can replace <code>IORef</code> with <code>TVar</code> or
similar objects for thread-safe implementations ;</li>
<li>several replicated functions mean several stores (can be useful in certain
cases).</li>
</ul>
<p>The huge drawback I see in that solution is its opacity. There’s also no way to
query the state of each cache. Such a feature could be added by proving a new
function, for instance. Same thing for deletion.</p>
<p>I’m one of those Haskellers who love purity. I try to keep my code the purest I
can, but there are exceptions, and that cache problem fits that kind of exception.</p>
<p>Feel free to comment, and as always, keep the vibe and happy hacking!</p>


guid: https://strongly-typed-thoughts.net/blog/zellij-2024Thu, 25 Apr 2024 10:43:00 GMT
title: Zellij is doing too much
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/zellij-2024
feedurl: https://phaazon.net/blog/feed
content:
<p>Today, I want to write about <a href="https://zellij.dev">Zellij</a> and especially its latest news update<a href="https://zellij.dev/news/welcome-screen-pipes-filepicker/">¹</a>.</p>
<h1>Zellij, the modern tmux?</h1>
<p>I came into Zellij a wild ago, while I was looking for something more modern over <a href="https://github.com/tmux/tmux/wiki">tmux</a>. It had the terminal
multiplexer features (windows, splits, focus, etc.), but was lacking features (sessions, for instance, were added
mid-2021<a href="https://github.com/zellij-org/zellij/blob/main/CHANGELOG.md#0120---2021-05-27">²</a>). As time passed, it started to get new features and promising. For instance, its layout system, allowing
to have a more minimalistic view (the default one being really all-over-the-place).</p>
<p>And then I realized that some very basic features from tmux were absent:</p>
<ul>
<li>When I focus on a pane (<code>prefix+z</code> by default in tmux; <code>ToggleFocusFullscreen</code> action in Zellij), in Zellij, I don’t
have any indication the pane is being focused, while tmux has this nice and neat <code>Z</code> marker in the statusbar. That
doesn’t seem like important, but it is, especially when you keep zooming in and out.</li>
<li>The statusbar… well, tmux has a very-well defined one, highly customizable. Zellij… doesn’t really have one. A
statusbar is just a plugin that runs on a 1-row layout. You can hide it by switching to a different layout, that
sounds overly complicated for such a basic feature.</li>
<li>Sessions cannot be renamed easily; you have to enter something they call the “session manager”.</li>
</ul>
<h2>Session manager, and the beginning of problems</h2>
<p>What I like about a software, like, for instance, <a href="https://github.com/tmux/tmux/wiki">tmux</a> or <a href="https://kakoune.org">kakoune</a>, is that they are <em>minimal</em>. They do one thing and
they do it well. If you are not sure about whether something is <em>minimal</em>, ask yourself a simple question: <em>“What do
we expect from this tool to solve; to do?”</em>. For a terminal multiplexer, we expect:</p>
<ul>
<li>To be able to split a window into panes; navigate the panes; focus them, etc.</li>
<li>To have several windows (tabs).</li>
<li>To have sessions.</li>
<li>To have a customizable statusbar.</li>
<li>To have some general purpose utilities, such as popups, etc.</li>
<li>To provide a nice API / RPC system to be able to control the system via scripting / hooks / etc.</li>
<li>That’s roughly all.</li>
</ul>
<p>If you provide a few amounts of nice features, and composability tools, then your users can build <em>more</em> by composing
your system with others. For instance, <a href="https://github.com/tmux/tmux/wiki">tmux</a> has the <code>display-popup</code> (<code>popup</code>) command that takes a command to run
and displays it in a floating window. That’s great, because now I can do something like <code>tmux ls -F '#S'</code> to list all
running sessions, pipe it to my favorite fuzzy picker (<code>fzf</code>, <code>sk</code>, etc.), and redirect the result to
<code>xargs tmux switch-client -t</code>. Something like:</p>
<pre><code class="language-sh">popup -E &quot;tmux ls -F '#S' | sk | xargs tmux switch-client -t&quot;
</code></pre>
<p>Make a keybinding for that, and here you go, you have a customized session picker!</p>
<p>On the other side, Zellij ships with its “session manager” — which, honestly looks terrible to me.</p>
<p><img
  src=https://zellij.dev/img/zellij-session-manager-animated.gif
	width=800
/></p>
<p>That thing is not something you have control on, and the Zellij CLI Actions<a href="https://zellij.dev/documentation/cli-actions#cli-actions">³</a> won’t help you either.</p>
<blockquote>
<p>Note: we are also in the right to ask ourselves why there is a list of commands (that you can bind to), and a list of
CLI actions. The former is richer than the latter and it means that the commands you bind to your keys and the
commands you run on the CLI do not share the same naming / they are not from the same set.</p>
</blockquote>
<p>So you’re stuck with a “native” session picker, that could have been pushed to the users. Again, with tmux, it’s a
one-liner, and it uses my favorite fuzzy picker.</p>
<h2>It gets worse</h2>
<p>Lately, I have decided to head over the Zellij news<a href="https://zellij.dev/news/">⁴</a> section to read about recent updates, and as mentioned at the
very top of this article, I was not disappointed. The latest update features some new enhancements, which goes against
the concept of minimalism and even against the initial scope of the project:</p>
<ul>
<li>A “welcome screen”, which is just something someone would create with a single script.</li>
<li>Strider, a file picker. I was so surprised to find that in a terminal multiplexer. I guess you don’t have to use it,
but the fact it’s there sounds very weird. Yet another thing to maintain in a larger and larger codebase. Hm.</li>
<li>They introduce <code>zpipe</code> to pipe results into plugins. That’s interesting, but that’s also a new burden to maintain
because of using plugins in the first place, instead of going the composability route. I’m not convinced. With tmux,
I can simply send the result of a command to a specific session, window, pane. All of that just looks like a
justification for the extra complexity added by plugins. But at least it makes sense once you have decided to use
plugins?</li>
</ul>
<h2>But it can be for you</h2>
<p>However, I don’t want to bash on Zellij. I think this very article I’m writing should be my reflection about why Zellij
exists, why it’s not for me (anymore?) and why it could be for you. See, tmux is an old, battle-tested piece of software
that will do exactly what it’s supposed to. If you want more, well, then you have to build it yourself. Because tmux
composes well, it’s an enjoyable process for me, but it might not be for you. Maybe you don’t want to have to learn a
fuzzy picker and the default, native session manager of Zellij will be a breeze for you. Maybe the file picker they have
made will be of great help for you.</p>
<p>I could summarize my thoughts as follows:</p>
<ul>
<li>If you are like me and like minimalism, UNIX philosophy and well-scoped softwares, then Zellij is definitly not a good
pick, and you will be better off it; go to tmux, a TWM, etc.</li>
<li>If you don’t care about all that philosophy and responsability stuff, then go for it! You’ll probably enjoy it for
being more user friendly!</li>
</ul>


guid: https://strongly-typed-thoughts.net/blog/luminance-0.27Thu, 28 Jun 2018 14:00:00 GMT
title: luminance-0.27: cleanup, easier to use and examples!
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/luminance-0.27
feedurl: https://phaazon.net/blog/feed
content:
<p><a href="https://crates.io/crates/luminance/0.27.1">luminance-0.27</a> was released lately and a bunch of new and interesting things was added.</p>
<h1>Easier API</h1>
<p>The <a href="https://github.com/phaazon/luminance-rs/blob/master/CHANGELOG.md#0270">CHANGELOG</a> is available to read but one of the most important change was the refactoring of
the pipeline system and of the overall API to make it less confusing. For instance, the
<code>Framebuffer::default</code> is now renamed <code>Framebuffer::back_buffer</code>. Another example is the <em>face
culling</em>, that was in pre-0.27 enabled by default (causing people who don’t now about it to be
confused because of a black screen). This is now disabled by default; hence less confusion.</p>
<p>The pipeline system types are now easier to use. For instance, redundancy was removed with the
various <code>Bound*</code> objects. When you want a <em>bound buffer of <code>f32</code></em> for instance, you just have to
use the type <code>BoundBuffer&lt;'a, f32&gt;</code> instead of the pre-0.27 <code>BoundBuffer&lt;'a, Buffer&lt;f32&gt;&gt;</code>. That is
a small change but it will eventually make working with luminance more comfortable.</p>
<p>Another change that I think is beneficial is the new <code>TessSlice</code> type – that replaces the pre-0.27
<code>TessRender</code>. This type represents a <em>GPU tessellation slice</em>. Such a slice can be used to render
part of a tessellation. What’s great is that the legacy methods to build such slices
(<code>TessRender::one_whole</code> for instance) have a new, more generalized way to do it:
<code>TessSliceIndex&lt;Idx&gt;</code>. This trait gives you a <code>slice</code> method which argument has type <code>Idx</code>. Several
implementors exist:</p>
<ul>
<li><code>impl TessSliceIndex&lt;Range&lt;usize&gt;&gt; for Tess</code>.</li>
<li><code>impl TessSliceIndex&lt;RangeFrom&lt;usize&gt;&gt; for Tess</code>.</li>
<li><code>impl TessSliceIndex&lt;RangeTo&lt;usize&gt;&gt; for Tess</code>.</li>
<li><code>impl TessSliceIndex&lt;RangeFull&lt;usize&gt;&gt; for Tess</code>.</li>
</ul>
<p>They all give you a <code>TessSlice</code> to work with (with the appropriate lifetime). The four
implementations listed above can be invoked with:</p>
<ul>
<li><code>tess.slice(0..10)</code>.</li>
<li><code>tess.slice(0..)</code>.</li>
<li><code>tess.slice(..10)</code>.</li>
<li><code>tess.slice(..)</code>.</li>
</ul>
<p>For those who wonder:</p>
<blockquote>
<p><em>Why have you not implemented <a href="https://doc.rust-lang.org/std/ops/trait.Index.html">Index</a> instead?</em></p>
</blockquote>
<p>The current <a href="https://doc.rust-lang.org/std/ops/trait.Index.html">Index</a> trait doesn’t give you enough power to index a value by returning something else
than a direct reference. Everything is summed up <a href="https://github.com/rust-lang/rfcs/pull/2473">in the RFC I wrote to fix Index</a>.</p>
<p>Feel free to have a look at the <a href="https://github.com/phaazon/luminance-rs/blob/master/CHANGELOG.md#0270">CHANGELOG</a> for further information.</p>
<h1>Examples, examples, examples!</h1>
<p>One of the most important feature people have asked is adding examples. The online documentation is
not enough – even if I spent lots of hours enhancing it. So I added several examples:</p>
<ul>
<li><strong>01-hello-world</strong>: learn how to draw two colored triangles by using vertex
colors (comes in <em>direct</em> and <em>indexed</em> geometry versions).</li>
<li><strong>02-render-state</strong>: learn how to change the render state to change the way the
triangles are rendered or how fragment blending happens.</li>
<li><strong>03-sliced-tess</strong>: learn how to slice a single GPU geometry to dynamically
select contiguous regions of it to render!</li>
<li><strong>04-shader-uniforms</strong>: send colors and position information to the GPU to
add interaction with a simple yet colorful triangle!</li>
<li><strong>05-attributeless</strong>: render a triangle without sending any vertex data to the
GPU!</li>
<li><strong>06-texture</strong>: learn how to use a loaded image as a luminance texture on the GPU!</li>
<li><strong>07-offscreen</strong>: get introduced to <em>offscreen rendering</em>, a powerful technique
used to render frames into memory without directly displaying them on your screen. Offscreen
framebuffers can be seen as a generalization of your screen.</li>
</ul>
<p>You can find them in <a href="https://github.com/phaazon/luminance-rs/blob/master/CHANGELOG.md">this place</a>.</p>
<p>Please do contribute if you thing something is missing, either by opening an issue or by opening a
PR. Any kind of contribution is highly welcomed!</p>


title: On dealing with owning and borrowing in public interfaces
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/on-owning-borrowing-pub-interface
feedurl: https://phaazon.net/blog/feed
guid:
https://strongly-typed-thoughts.net/blog/on-owning-borrowing-pub-interfaceSun, 04 Nov 2018 01:30:00 GMT

content:
<blockquote>
<p>Disclaimer: this blog is about <a href="https://www.rust-lang.org">Rust</a> and some of its intrinsics semantics, along with software
design and architecture – especially for public interfaces like APIs. However, you may find
it interesting to apply the concepts to any language that would support those ideas directly or
indirectly.</p>
</blockquote>
<p>I’ve been writing on a few examples code lately to add to documentations of some crates of mine. I
write a lot of code that creates new objects that need other objects in order to be built. Most of
the APIs you can see around tend to love the borrow principle – and I do. The idea is simple:</p>
<ul>
<li>If you want to create a new type that, for instance, must own a <code>String</code>, most APIs tend to
agree that the constructor of your type should take a <code>&amp;str</code> or some sort of borrowing – typical
way to do that is to use the <code>AsRef</code> trait, that gives you more power (more about that below).
You can then just clone the <code>&amp;str</code> in your function.</li>
<li>Apply that principle to pretty much any kind of object you need but not types that don’t have
easy borrowing semantics. For instance, <code>Rc&lt;_&gt;</code> already carries sharing and dynamic borrowing
semantics, so I wouldn’t be surprised to find a constructor that takes a <code>Rc&lt;_&gt;</code> instead of a
<code>&amp;Rc&lt;_&gt;</code>.</li>
</ul>
<p>Depending on the crate you look at, the authors and how they see things, you might find a lot of
ways to pass that string to your constructor. Let’s get technical. Especially, I want this blog post
to give people a hint at how they should shape their APIs by driving the types with semantics.</p>
<h1>The base code</h1>
<pre><code>struct Person {
  ident: String
}
</code></pre>
<p>Very simple to get started. We have a <code>Person</code> type that carries the name of that person. We don’t
want to expose the internals of the type because we might add a lot of other things in there and
allowing people to pattern-match a <code>Person</code> would break their code when we add more things.</p>
<pre><code>impl Person {
  pub fn new(name: String) -&gt; Person {
    Person { name }
  }
}
</code></pre>
<p>That works. This constructor works by <em>move semantics</em>: it expects you to move a <code>String</code> in in
order to copy it into the <code>Person</code> value. Move semantics are clear here: the <em>client</em> must allocate
a <code>String</code> or move one already allocated by someone else. This can get a bit boring. Imagine:</p>
<pre><code>let someone = Person::new(&quot;Patrick Bateman&quot;); // this won’t compile
</code></pre>
<p><code>Person::new(&quot;Patrick Bateman&quot;)</code> doesn’t typecheck because <code>&quot;Patrick Bateman&quot;</code> has type <code>&amp;'static str</code> here. It’s not a <code>String</code>.</p>
<h1>Drive the allocation from within your code</h1>
<p>So how can we fix the code above to have it compile?</p>
<pre><code>let someone = Person::new(&quot;Patrick Bateman&quot;.to_owned()); // okay now
</code></pre>
<p><code>&quot;Patrick Bateman&quot;.to_owned()</code> makes a use of
<a href="https://doc.rust-lang.org/std/borrow/trait.ToOwned.html#tymethod.to_owned"><code>ToOwned::to_owned</code></a> to
clone the string. You could also have used
<a href="https://doc.rust-lang.org/std/convert/trait.Into.html"><code>Into::into</code></a>. However, we’re still
requiring clients of our API to perform the allocation – or move in, which is not ideal. A typical
solution to this is to use a borrow and clone when needing an owned version.</p>
<pre><code>impl Person {
  pub fn new(name: &amp;str) -&gt; Person {
    Person {
      name: name.to_owned()
    }
  }
}
</code></pre>
<p>This API enables the current code to compile:</p>
<pre><code>let someone = Person::new(&quot;Patrick Bateman&quot;); // okay because 'static is a subtype of 'a
</code></pre>
<p>We’re all good now, right? Not exactly: what happens if we try to pass the initial <code>String</code> we had
when we moved something in?</p>
<pre><code>let x: String = …;
let someone = Person::new(x); // not okay now since it’s a type mismatch
</code></pre>
<p>To fix this, we need to pass a <code>&amp;str</code> out of a <code>String</code>. Since
<a href="https://doc.rust-lang.org/std/ops/trait.Deref.html#impl-Deref-3"><code>String</code> implements <code>Deref&lt;Target = str&gt;</code></a>,
it’s quite straight-forward:</p>
<pre><code>let x: String = …;
let someone = Person::new(&amp;x); // okay, here &amp;x derefs to &amp;str
</code></pre>
<blockquote>
<p>You can also use the more explicit <code>String::as_str</code> method.</p>
</blockquote>
<p>A nice trick here to be able to pass both <code>&amp;str</code> and <code>String</code> is to use the <a href="https://doc.rust-lang.org/std/convert/trait.AsRef.html"><code>AsRef</code></a> trait.</p>
<pre><code>impl Person {
  pub fn new&lt;N&gt;(name: N) -&gt; Person where N: AsRef&lt;str&gt; {
    Person {
      name: name.as_ref().to_owned()
    }
  }
}
</code></pre>
<p>In this case, <code>AsRef&lt;str&gt;</code> is implemented for both <code>&amp;str</code> (it just returns itself) and <code>String</code> – it
just deref / uses the <code>as_str</code> method).</p>
<p>However, you can still see a problem here. If we keep using <code>x</code> after the call to <code>Person::new</code>,
this code is actually okay and we can move on. If we don’t need <code>x</code> afterwards, we’re just wasting
an opportunity to move in instead of allocating!</p>
<h1>Drive the allocation from your code and allow for moving in</h1>
<p>Clearly, to me, the perfect API would enable you to pass borrows and ask the API code to clone for
you or just accept the memory region you provide (i.e. you move something in). The idea is then to
accept either <code>&amp;str</code> or <code>String</code> in our case, and clone only a <code>&amp;str</code>… There are several traits and
types that provide that feature.</p>
<h2><code>ToOwned</code> + <code>Cow</code></h2>
<p><a href="https://doc.rust-lang.org/std/borrow/enum.Cow.html"><code>Cow</code></a> – for Clone on write – is a very interesting concept. It encodes that you’re either
borrowing some data or that you’ve already borrowed it. See it as:</p>
<pre><code>enum Cow&lt;'a, T&gt; where T: 'a + ?Sized + ToOwned {
  Borrowed(&amp;'a T),
  Owned(T::ToOwned)
}

impl&lt;'a, T&gt; Cow&lt;'a, T&gt; where T: 'a + ?Sized + ToOwned {
  pub fn into_owned(self) {
    match self {
      Cow::Borrowed(b) =&gt; b.to_owned(),
      Owned(o) =&gt; o
    }
  }
}
</code></pre>
<p>Now, the interesting part: it’s possible to go from <code>&amp;'a str</code> to <code>Cow&lt;'a, str&gt;</code> and from <code>String</code>
to <code>Cow&lt;'a, str&gt;</code> by using <code>Into</code> implementors. That enables us to write this:</p>
<pre><code>impl Person {
  pub fn new&lt;'a, N&gt;(name: N) -&gt; Person where N: Into&lt;Cow&lt;'a, str&gt;&gt; {
    Person { name: name.into().into_owned() }
  }
}
</code></pre>
<p>This code will move in a <code>String</code> if you passed one and clone if you passed <code>&amp;str</code>. The following
lines of code compile and work as a charm:</p>
<pre><code>let _ = Person::new(&quot;Patrick Bateman&quot;);

let dawg = &quot;Dawg&quot;;
let _ = Person::new(format!(&quot;Doggo {}&quot;, dawg));
</code></pre>
<p>What’s interesting here is that all cases are covered:</p>
<ul>
<li>The client <strong>has the choice</strong> to either allocate – or move in – or borrow.</li>
<li>It’s even possible to implement <code>Into&lt;Cow&lt;str&gt;&gt;</code> on other types to pass other kind of arguments.</li>
</ul>
<p>There’s just a little nit: <code>Cow::into_owned</code> obviously patterns match on the variant, inducing a
small yet present runtime overhead. We tend to prefer using <code>Cow&lt;_&gt;</code> to dynamically dispatch the
decision to clone at runtime, while in our case, it’s more about a static choice (which API function
version to use).</p>
<h2><code>Into</code>, as simple as it gets</h2>
<p>If you look at the <code>Into&lt;String&gt;</code> implementors, you’ll find <code>impl Into&lt;String&gt; for String</code> –
actually, this is a
<a href="https://doc.rust-lang.org/std/convert/trait.From.html#impl-From%3CT%3E-10">blanket implementor</a> –
and <code>impl&lt;'a&gt; Into&lt;String&gt; for &amp;'a str</code>. That implements the same semantics as <code>Cow&lt;str&gt;</code>, but at
the type level, removing any remaining runtime overhead.</p>
<p>The new code is even simpler:</p>
<pre><code>impl Person {
  pub fn new&lt;N&gt;(name: N) -&gt; Person where N: Into&lt;String&gt; {
    Person { name: name.into() }
  }
}
</code></pre>
<p>Obviously, there are drawbacks:</p>
<ul>
<li>You cannot express any lifetimes with this <code>Into&lt;String&gt;</code> trait bound. If you need, for any
reason, to dynamically check whether to clone or not, the <code>Cow&lt;str&gt;</code> is the right decision. If
you know you’ll always need to allocate, go for <code>Into&lt;String&gt;</code>.</li>
<li><code>Into&lt;String&gt;</code> is vague and some types might not even implement it.</li>
<li><code>Into&lt;String&gt;</code> doesn’t allow for cheap reading while <code>Cow&lt;str&gt;</code> does. <code>Into&lt;String&gt;</code> requires
you to move or allocate prior to reading.</li>
</ul>
<h1>Let’s draw a conclusion</h1>
<p>What I wanted to highlight in this blog post is that <code>&amp;_</code>, <code>AsRef&lt;_&gt;</code>, <code>Cow&lt;_&gt;</code>, and <code>Into&lt;_&gt;</code> all
have different semantics that can be used to <em>encode</em> different contracts in public interfaces.</p>
<ul>
<li><code>&amp;T</code> means that you don’t require your client to clone nor move because you <em>might</em> only perform
read-only computations on <code>T</code>. That gives some intuitions to your clients:
<ul>
<li>You don’t require them to own <code>T</code>.</li>
<li>If any allocation must be performed, the client doesn’t have to worry about it.</li>
<li>You force them to use a reference. It can have interesting consequences. Imagine
<code>&amp;[Something]</code>. This carries one information more: contiguity of the input data – slices are
contiguous.</li>
</ul>
</li>
<li><code>Q: AsRef&lt;T&gt;</code> is a <code>&amp;T</code> on steroids in terms of software design. It gives more power to the
client as they’ll be able to pass more types in your functions. However, you now introduce a
polymorphic API. The semantics are the same: you plan to perform only read-only computations or
accept not to use a client-provided move-in value if you need to own something. Keep in mind an
important <em>hidden property</em> here: because you accept values to be moved in while you just have
a read-only contract on them, you also accept clients values to be <em>dropped</em> by corollary.</li>
<li><code>Cow&lt;T&gt;</code> states that the client can provide a borrow or an owned variable and you will take
advantage of that <strong>at runtime</strong>. This is important as it gives a good idea about how the memory
footprint of the function works: it will take decision at runtime whether or not it needs to own
memory.</li>
<li><code>Q: Into&lt;T&gt;</code> has vague and large semantics but can be used to encode the fact that you <em>want</em>
owned data, but you accept your clients to provide a borrow as long as you can clone. If they
provide you with an owned data, you’ll just do nothing (move in).</li>
<li>(bonus) <code>Q: IntoIterator&lt;Item = T&gt;</code> is also something you can use for the slices thing just
above. If (and only if) you don’t care about contiguity or will just inspect values one by one,
this is the interface to go. This interface gives hints that your function might iterate through
the input and perform (perhaps heavy) computation on each items before stepping to the next one.
Using <code>IntoIterator</code> instead of <code>Iterator</code> enables you the same kind of trick you have with
<code>AsRef&lt;_&gt;</code> vs. <code>&amp;_</code>: you can take a <code>Vec&lt;_&gt;</code> directly instead of the iterator directly. Keep in
mind that this might not have sense for some types, especially if they have several iterator
interfaces. The <code>str</code> type has for instance the
<a href="https://doc.rust-lang.org/std/primitive.str.html#method.bytes">str::bytes</a> method that gives
you an iterator over bytes and the
<a href="https://doc.rust-lang.org/std/primitive.str.html#method.chars">str::chars</a> method that yields
an iterator over UTF-8 characters.</li>
</ul>
<p>This list gives you a good idea about what interface you should use in your public interfaces.
Read-only? Owned data? Read and maybe write? All those semantics are visible through those types and
traits, you should definitely try to wrap your finger around using all of them. Of course, if you’re
just writing a small utility function that needs to borrow the <code>.name</code> of a <code>Person</code>, passing in a
<code>&amp;Person</code> seems completely sound. Most of the time, if you don’t know where the data comes from,
be the more inclusive and generic as possible.</p>
<p>I hope you like that small article, and as always, keep the vibes.</p>


guid: https://strongly-typed-thoughts.net/blog/al_0.1_releasedSat, 28 Feb 2015 00:00:00 GMT
title: al 0.1 released!
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/al_0.1_released
feedurl: https://phaazon.net/blog/feed
content:
<h1>What is OpenAL?</h1>
<p><img src="http://upload.wikimedia.org/wikipedia/en/thumb/2/28/OpenAL_logo.png/200px-OpenAL_logo.png" alt="" /></p>
<p><a href="http://www.openal.org/">OpenAL</a> is an open-source and free audio library
developped by <strong>Creative Technology</strong>. The first version, however, was made by
<strong>Loki Software</strong>. The API targets spacialized audio and sound managing so that
it’s simple to simulate sound attenuation over distance and
<a href="http://en.wikipedia.org/wiki/Doppler_effect">Doppler effect</a>. <strong>OpenAL</strong> is
supported on a <a href="http://en.wikipedia.org/wiki/OpenAL#Supported_platforms">wide variety of platforms</a>,
from PC (Windows, Linux, FreeBSD, Mac OSX and so on) to consoles (PS3, Xbox…).</p>
<p>The latest version was released in 2005 (<em>9 years old ago!</em>) and is OpenAL 1.1.
It’s then pretty logical that we should have OpenAL 1.1 in <strong>Haskell</strong>.</p>
<h1>OpenAL and Haskell</h1>
<blockquote>
<p><em>“Wait… we already have <a href="https://hackage.haskell.org/package/OpenAL">OpenAL</a>
in Haskell!”</em></p>
</blockquote>
<p>That’s true. However, the <a href="https://hackage.haskell.org/package/OpenAL">OpenAL</a>
Haskell package has some serious issues.</p>
<p>The first one is that package is not a direct binding to <strong>OpenAL</strong>. It’s a
high-level wrapper that wraps stuff with obscure types, like <code>StateVar</code> or
<code>ObjectName</code>. Even though such abstractions are welcomed, we should have a raw
binding so that people who don’t want those abstractions can still use
<strong>OpenAL</strong>. Moreover, such high libraries tend to lift resources allocations /
deallocations into the GC[^GC]. That is a pretty bad idea because we,
sometimes, want to control how long objects live.</p>
<p>The second one, which is as terrible, is the fact that those abstractions aren’t
defined by the Haskell <a href="https://hackage.haskell.org/package/OpenAL">OpenAL</a>
package, but by another, which is…
<a href="https://hackage.haskell.org/package/OpenGL">OpenGL</a>. The latter is <strong>not</strong> a
raw binding to <strong>OpenGL</strong> – for that have a look at
<a href="https://hackage.haskell.org/package/OpenGLRaw">OpenGLRaw</a> or
<a href="https://hackage.haskell.org/package/gl">gl</a> – but the same kind of high-level
oriented package. A lot of us have asked the authors of both the package to
agree of making those abstractions shared between both the libraries; they
never seemed to agree.</p>
<p>And even though, having a dependency between <strong>OpenAL</strong> and <strong>OpenGL</strong> is
insane!</p>
<p>Sooooo… since <a href="https://github.com/ekmett">Edward Kmett</a> had the idea of
<a href="https://hackage.haskell.org/package/gl">gl</a> to fix similar issues, I think I
should have the idea of <a href="https://hackage.haskell.org/package/al">al</a> to go along
with <a href="https://hackage.haskell.org/package/gl">gl</a>. The idea is that if you want
to use one, you can use the other one without any conflict.</p>
<h1>al 0.1</h1>
<p>Yesterday night, around 20:00, I decided to make it. It took me several hours
to write the raw binding, but I finally made it and released
<a href="https://hackage.haskell.org/package/al">al 0.1</a> the very next day – uh, today!</p>
<p>Because <strong>OpenAL</strong> is not shipped with your [^OS] nor anything special, and thus
because you have to explicitely install it, installing
<a href="https://hackage.haskell.org/package/al">al</a> requires a bit more than just
typing <code>cabal install al</code>.</p>
<h2>Installing al</h2>
<h3>cabal update</h3>
<p>First things first:</p>
<pre><code>cabal update
</code></pre>
<p>You should have <a href="https://hackage.haskell.org/package/al">al</a> available if
you type <code>cabal info al</code> afterwards.</p>
<h3>c2hs</h3>
<p><a href="https://hackage.haskell.org/package/al">al</a> requires you to have
<a href="https://hackage.haskell.org/package/c2hs-0.23.1">c2hs</a> – version <strong>0.23.*</strong> –
installed. If not:</p>
<pre><code>cabal install c2hs
</code></pre>
<p>Be sure it’s correctly installed:</p>
<pre><code>c2hs --version
C-&gt;Haskell Compiler, version 0.23.1 Snowbounder, 31 Oct 2014
  build platform is &quot;i386-mingw32&quot; &lt;1, True, True, 1&gt;
</code></pre>
<h3>OpenAL SDK</h3>
<p>Of course, you should have the <strong>OpenAL SDK</strong> installed as well. If not, install
it. It should be in your package manager if you’re on Linux / FreeBSD, and you
can find it <a href="http://www.openal.org/creative-installers/">here</a> for Windows
systems.</p>
<blockquote>
<p><strong>Important: write down the installation path, because we’ll need it!</strong></p>
</blockquote>
<h3>Installing al</h3>
<p>Here we are. In order to install, <a href="https://hackage.haskell.org/package/al">al</a>
needs the paths to your <strong>OpenAL SDK</strong>. I’m not sure whether it could vary a
lot, though. The default Windows paths are:</p>
<ul>
<li><code>C:\Program Files (x86)\OpenAL 1.1 SDK\libs</code> for the libraries ;</li>
<li><code>C:\Program Files (x86)\OpenAL 1.1 SDK\include</code> for the header files.</li>
</ul>
<p>In future releases, I’ll default to those so that you don’t have to explicitely
pass the paths if you have a default installation. But for the moment, you have
to pass those paths when installing:</p>
<pre><code>cabal install al --extra-lib-dirs=&quot;C:\Program Files (x86)\OpenAL 1.1 SDK\libs&quot;
--extra-include-dirs=&quot;C:\Program Files (x86)\OpenAL 1.1 SDK\include&quot;
</code></pre>
<p>Of course, you may adapt paths to your situation. You may even try without that
for UNIX systems – it might already be in your <code>PATH</code>, or use slashs for
<strong>MinGW</strong>.</p>
<h2>Using al</h2>
<p><a href="https://hackage.haskell.org/package/al">al</a> lays into two module trees:</p>
<ul>
<li><code>Sound.AL</code></li>
<li><code>Sound.ALC</code></li>
</ul>
<p>The former exports anything you might need to use the <strong>OpenAL API</strong> while
the latter exports symbols about the <strong>OpenAL Context API</strong>. Please feel free
to dig in each tree to import specific types or functions.</p>
<h2>OpenAL documentation</h2>
<p><a href="https://hackage.haskell.org/package/al">al</a> doesn’t have any documentation for
a very simple reason: since it’s a raw binding to the <strong>C OpenAL API</strong>, you
should read the <strong>C</strong> documentation. Translating to <strong>Haskell</strong> is
straightforward.</p>
<p><a href="http://www.openal.org/documentation/openal-1.1-specification.pdf">The OpenAL 1.1 Specifications</a></p>
<h1>What’s next?</h1>
<p>Up to now, <a href="https://hackage.haskell.org/package/al">al</a> doesn’t cover
extensions. It’s not shipped with <strong>ALUT</strong> either as I want to keep the package
low-level and raw – and don’t even try to make me change my mind about that ;) .</p>
<p>If you have any issues, please let me know on the issues tracker. I’m responsive
and I’ll happily try to help you :) .</p>
<p>Once again, have fun, don’t eat too much chocolate nor sweets, and happy Easter
Day!</p>
<p>[^GC]: Garbage Collector
[^OS]: Operating System</p>


guid: https://strongly-typed-thoughts.net/blog/neovim-plugins-stabilityWed, 03 Nov 2021 00:24:00 GMT
title: Neovim plugins stability
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/neovim-plugins-stability
feedurl: https://phaazon.net/blog/feed
content:
<p>Have you ever installed a Neovim plugin and realized a few days / weeks later, after updating the
plugin, that something broke? That you now have have annoying error messages at startup? If so, then
this blog article should ring a bell. If not, it will probably open your eyes on a problem that can
bite you at any moment.</p>
<h1>Neovim and plugins</h1>
<p>I have been in that situation where I struggle with plugins breaking every time I update, and not
only in Neovim. The problem can occur pretty much anywhere. The main reason to this is simple:
<em>breaking changes</em> happen, and must happen. Plugins maintainers might decide at some point that
something must be removed, or changed. A configuration option you were using disappears or its
name changes. That happens and it’s part of the how software works.</p>
<p>However, there is a difference between breaking changes in plugins and breaking changes in i.e.
libraries / binaries we write in languages supporting package managers, such as Rust and <code>cargo</code>,
Haskell and <code>cabal</code>, Python and <code>poetry</code>, etc. etc. Actually, there are two main differences,
depending on the perspective you take to look at the problem:</p>
<ul>
<li>From a user perspective, software is not versioned and « they want the last updated thing. »
Obviously, if you are a programmer, you know that it’s likely something is versioned, but most
of the time, when you install something like a video game you like, an editor, a plugin or
anything, do you really ask for a specific version? There are obviously situations where you
need to ensure the versions that you use, but most of the time, <em>people do not care</em>.</li>
<li>From a plugin author perspective, there is no tooling to ensure that you can ship a feature
without breaking your users.</li>
</ul>
<p>About the last point, when I work on a given software project, I specify the dependencies of my
project, and I specify the versions of those dependencies. Using <a href="https://semver.org">SemVer</a>, I’m sure that if the
those dependencies are correctly written, I should be able to release my software (lib or bin)
without any issues on my users. There are exceptions, but those are rare and most of the time
related to human errors when implementing <a href="https://semver.org">SemVer</a> incorrectly.</p>
<p>When you write a plugin, you will most likely write it in a language and ecosystem that doesn’t even
support versioning. Which brings me to the main point of this article: how do you prevent breaking
your users?</p>
<h1>The git channel and the « latest » problem</h1>
<p>Neovim plugins are often implemented in package managers like
<a href="https://github.com/wbthomason/packer.nvim">packer</a> via Git URLs. Most of the time, <code>foo/bar</code> will be installed as the <code>bar</code> plugin, part of
the org / user <code>foo</code> on GitHub. You can also install local plugins but that’s off topic here. When
you provide the description of the plugin to install, you will most of the time provide nothing more.
This will default to install and synchronize the plugin using the <code>master</code> branch (or whatever your
local <code>git</code> says, like <code>main</code>). As a Software Engineer, to me, this is a really bad habit we all
have, and I’ll explain why.</p>
<p>When you depend on the <em>latest</em> version of something, you are basically signing a contract between
your local system and the remote system to keep your local version in sync with remote. If the
remote gets updated, you get the changes the next time you update. The problem with this is that
<code>master</code> contains zero information about whether an update is a breaking change or not. It’s the
same reason why defaulting API calls to your API service to the latest version of the API is <em>wrong</em>
and that you should default them to <code>v1</code>: because it will never change, so people have to <em>opt-in</em>
for breaking changes instead, which is a much saner way of dealing with breaking changes.</p>
<p>So I’ve been thinking about this problem for quite a while now. In ecosystem like Rust, crates have
to implement <a href="https://semver.org">SemVer</a>, so that <code>cargo</code>, the package manager and build system, knows how to resolve
the dependency graph given your version requirements. But we don’t have that with Neovim plugins… or
do we?</p>
<h1>The solution is (partially) already there</h1>
<p>In <a href="https://github.com/wbthomason/packer.nvim">packer</a> (and probably others), we can provide additional information to download (and update)
plugins. For instance, <a href="https://github.com/wbthomason/packer.nvim">packer</a> supports the <code>branch = …</code>, <code>tag = …</code> and <code>commit = …</code> configuration
options. So this gave me an idea. And this idea is something I would like to turn into a proposal
for the Neovim community. The idea is to use both branches and tags to encode <a href="https://semver.org">SemVer</a> information
and allow users to specify plugins a bit like you would specify dependencies in tools like <code>cargo</code>.
In order to do that, plugin authors must accept and implement <a href="https://semver.org">SemVer</a>. The current way I see things
is actually pretty simple:</p>
<ul>
<li><code>master</code>, <code>main</code>, etc. can be used as a <em>nightly</em> channel. Users who don’t want to deal with this
<a href="https://semver.org">SemVer</a> idea can just ignore the whole thing and still use their plugins the way they’ve always
had.</li>
<li>A triplet git tag, <code>vM.m.p</code>, represents a full, immutable version that cannot be changed. This
would allow a user to depend on <code>vM.m.p</code> and be sure that the plugin would be pinned to that
version. Here, <code>M</code> is the major version, <code>m</code> is the minor version and <code>p</code> is the patch version.
Every time the plugin is modified in a way that doesn’t show on the interface (no breaking change
and no non-breaking change, only internal patches), the <code>p</code> version is incremented.</li>
<li>A pair git branch <code>vM.m</code>, with <code>M</code> the major and <code>m</code> the minor version. The idea is that every
time a plugin gets a new non-breaking change, the <code>m</code> version is incremented. That would allow
users to depend on <code>vM.m</code> to be sticky to that minor version, but still receive patches updates.
More on that below.</li>
<li>A single version git branch, <code>vM</code>, with <code>M</code> the major version. Every time the plugin gets a new
breaking change, <code>M</code> is incremented. This would probably be the most interesting options for
users, as it would allow them to benefit from new features without having their plugin break on an
update.</li>
</ul>
<p>In order to implement this scheme using git branches and git tags, there is one trick to implement.
One a plugin author decide to make a new release, they have to do several things:</p>
<ul>
<li>Create a git tag. If the previous version was <code>v1.2.3</code>, they will either create <code>v1.2.4</code>, <code>v1.3.0</code>
or <code>v2.0.0</code>, depending on the kind of change.</li>
<li>If they have created a new patch version:
<ul>
<li>They need to create the tag <code>v1.2.4</code>.</li>
<li>They need to update the branch <code>v1.2</code> so that it now points to the same commit as <code>v1.2.4</code>.</li>
<li>They need to update the branch <code>v1</code> so that it now points to the same commit as <code>v1.2.4</code>.</li>
</ul>
</li>
<li>If they have created a new minor version:
<ul>
<li>They need to create the tag <code>v1.3.0</code>.</li>
<li>They need to create the branch <code>v1.3</code> and make it point to the same commit as <code>v1.3.0</code>.</li>
<li>They need to update the branch <code>v1</code> so that it now points to the same commit as <code>v1.3.0</code>.</li>
</ul>
</li>
<li>IF they have created a new major version:
<ul>
<li>They need to create the tag <code>v2.0.0</code>.</li>
<li>They need to create the branch <code>v2.0</code> and make it point to the same commit as <code>v2.0.0</code>.</li>
<li>They need to create the branch <code>v2</code> and make it point to the same commit as <code>v2.0.0</code>.</li>
</ul>
</li>
</ul>
<p>With this approach, a user can now depend on a version and be sure not to get breaking-changes. An
example for my config for <a href="https://github.com/phaazon/hop.nvim">hop.nvim</a>:</p>
<pre><code class="language-lua">use {
  'phaazon/hop.nvim',
  branch = &quot;v1&quot;,
  config = function()
    require'hop'.setup {
      keys = 'etovxqpdygéèfblzhckisuran',
    }
  end
}
</code></pre>
<h1>Tooling</h1>
<p>I’m currently writing a (small) plugin, branching on package managers such as <a href="https://github.com/wbthomason/packer.nvim">packer</a>, to be able,
from a user perspective, to get a listing of packages that can be upgraded. For instance, if you
depend on <code>telescope.nvim-0.4</code>, if <code>telescope.nvim-0.4.9</code> is released, <a href="https://github.com/wbthomason/packer.nvim">packer</a> should pick it up
because you would have <code>branch = v0.4</code> in your config (and that branch would be updated to point to
git tag <code>v0.4.9</code>). However, if <code>telescope-nvim-0.5</code> is released, that tool will provide you with a
hint that a new version is available and that you need to manually select it, because it might break
your configuration.</p>
<h1>Pros. &amp; cons.</h1>
<p>The obvious advantage (and incentive) here is that if plugin authors accept to implement something
like this, plugin stability will be an old bitter memory. People who don’t care and want to live on
the bleeding edge can completely ignore this proposal and still use the <code>master</code> / <code>main</code> branch
(not providing the branch unfortunately often defaults to the <code>master</code> branch). The other incentive
here is that plugins (like the one I’m writing for the tooling) can now do more things with git tags
and branches to display more information, like the number of releases, git tag annotations to show
proper release notes inside Neovim, etc.</p>
<p>The drawbacks are not negligible: implementing such a <a href="https://semver.org">SemVer</a> proposal using git tags and branches
is going to require a bit more work, and the tooling is clearly lacking. This is very similar to how
dynamic relocatable objects (<code>.so</code>) are handled on most Linux systems: the actual <code>.so</code> file is
versioned somewhere on your file system, like <code>/usr/lib/foo.1.2.3.so</code>, and applications / other
libraries can depend on major and minor versions by using <em>symlinks</em>: <code>/usr/lib/foo.1.so</code> points to
<code>/usr/lib/foo.1.2.so</code>, which in turns points to <code>/usr/lib/foo1.2.3.so</code>. <code>git</code> doesn’t support sticky
branches (i.e. you can’t ask to make a branch reference another one, it has to reference a commit,
which is always immutable), so it means that updating a version requires you to update the whole
hierarchy of branches, which is a bit annoying. A tool (like a shell tool in the plugin I’m writing)
could probably help with that.</p>
<p>So what do you think? Worth it? I think that in terms of stability, it is a missing piece of pretty
much any editor supporting dynamic plugins. The « ideal » situation would be to support semantic
versioning directly at the package level (which is not <a href="https://github.com/wbthomason/packer.nvim">packer</a>, but directly how Neovim represents
package), and the actual encoding of the package versions would probably be hard to implement using
only a centralized system like GitHub as there is no index (besides git tags and branches).</p>


guid: https://strongly-typed-thoughts.net/blog/introducing_luminanceFri, 24 Jul 2015 00:00:00 GMT
title: Introducing Luminance, a safer OpenGL API
author: Dimitri Sabadie <dimitri.sabadie@gmail.com>
url: https://strongly-typed-thoughts.net/blog/introducing_luminance
feedurl: https://phaazon.net/blog/feed
content:
<p>A few weeks ago, I was writing <strong>Haskell</strong> lines for a project I had been
working on for a very long time. That project was a 3D engine. There are several
posts about it on my blog, feel free to check out.</p>
<p>The thing is… Times change. The more it passes, the more I become mature in what
I do in the <strong>Haskell</strong> community. I’m a demoscener, and I need to be
productive. Writing a whole 3D engine for such a purpose is a good thing, but I
was going round and round in circles, changing the whole architecture every now
and then. I couldn’t make my mind and help it. So I decided to stop working on
that, and move on.</p>
<p>If you are a <strong>Haskell</strong> developer, you might already know
<a href="https://www.fpcomplete.com/user/edwardk?show=all">Edward Kmett</a>. Each talk with
him is always interesting and I always end up with new ideas and new knowledge.
Sometimes, we talk about graphics, and sometimes, he tells me that writing a
3D engine from scratch and release it to the community is not a very good move.</p>
<p>I’ve been thinking about that, and in the end, I agree with Edward. There’re two
reasons making such a project hard and not interesting for a community:</p>
<ol>
<li>a good “3D engine” is a specialized one – for FPS games, for simulations,
for sport games, for animation, etc. If we know what the player will do, we
can optimize a lot of stuff, and put less details into not-important part
of the visuals. For instance, some games don’t really care about skies, so
they can use simple skyboxes with nice textures to bring a nice touch of
atmosphere, without destroying performance. In a game like a flight
simulator, skyboxes have to be avoided to go with other techniques to provide
a correct experience to players. Even though an engine could provide both
techniques, apply that problem to almost everything – i.e. space partitionning
for instance – and you end up with a nightmare to code ;</li>
<li>an engine can be a very bloated piece of software – because of point <em>1</em>.
It’s very hard to keep an engine up to date regarding technologies, and
make every one happy, especially if the engine targets a large audience of
people – i.e. <a href="http://hackage.haskell.org">hackage</a>.</li>
</ol>
<p>Point <em>2</em> might be strange to you, but that’s often the case. Building a
flexible 3D engine is a very hard and non-trivial task. Because of point <em>1</em>,
you utterly need to restrict things in order to get the required level of
performance or design. There are people out there – especially in the demoscene
world – who can build up 3D engines quickly. But keep in mind those engines are
limited to demoscene applications, and enhancing them
to support something else is not a trivial task. In the end, you might end up
with a lot of bloated code you’ll eventually zap later on to build something
different for another purpose – eh, demoscene is about going dirty, right?! ;)</p>
<h1>Basics</h1>
<p>So… Let’s go back to the basics. In order to include everyone, we need to
provide something that everyone can download, install, learn and use. Something
like <a href="https://www.opengl.org/sdk">OpenGL</a>. For <strong>Haskell</strong>, I highly recommend
using <a href="https://hackage.haskell.org/package/gl">gl</a>. It’s built against the
<code>gl.xml</code> file – released by Khronos. If you need sound, you can use the
complementary library I wrote, using the same name convention,
<a href="https://hackage.haskell.org/package/al">al</a>.</p>
<p>The problem with that is the fact that <strong>OpenGL</strong> is a low-level API. Especially
for new comers or people who need to get things done quickly. The part that
bothers – wait, no, <em>annoys</em> – me the most is the fact that <strong>OpenGL</strong> is a very
old library which was designed two decades ago. And we suffer from that. A lot.</p>
<p><strong>OpenGL</strong> is a <em>stateful</em> graphics library. That means it maintains a <em>state</em>, a
<em>context</em>, in order to work properly. Maintaining a context or state is a legit
need, don’t get it twisted. However, if the design of the API doesn’t fit such a
way of dealing with the state, we come accross a lot of problems. Is there <em>one</em>
programmer who hasn’t experienced black screens yet? I don’t think so.</p>
<p>The <strong>OpenGL</strong>’s API exposes a lot of functions that perform <em>side-effects</em>.
Because <strong>OpenGL</strong> is weakly typed – almost all objects you can create in
<strong>OpenGL</strong> share the same <code>GL(u)int</code> type, which is very wrong – you might end
up doing nasty things. Worse, it uses an internal binding system to <em>select</em> the
objects you want to operate on. For instance, if you want to upload data to a
texture object, you need to <em>bind</em> the texture before calling the texture upload
function. If you don’t, well, that’s bad for you. There’s no way to verify code
safety at compile-time.</p>
<p>You’re not convinced yet? <strong>OpenGL</strong> doesn’t tell you directly how to change
things on the GPU side. For instance, do you think you have to <em>bind</em> your vertex
buffer before performing a render, or is it sufficient to <em>bind</em> the <em>vertex array
object</em> only? All those questions don’t have direct answers, and you’ll need to
dig in several wikis and forums to get your answers – the answer to that question
is <em>“Just bind the VAO, pal.”</em></p>
<h1>What can we do about it?</h1>
<p>Several attempts to enhance that safety have come up. The first thing we
<strong>have</strong> to do is to wrap all <strong>OpenGL</strong> object types into proper types. For
instance, we need several types for <code>Texture</code> and <code>Framebuffer</code>.</p>
<p>Then, we need a way to ensure that we cannot call a function if the context is
not setup for. There are a few ways to do that. For instance,
<a href="https://hackage.haskell.org/package/indexed-0.1">indexed monads</a> can be a good
start. However, I tried that, and I can tell you it’s way too complicated. You
end up with very long types that make things barely unreadable.
<a href="https://github.com/phaazon/igl/blob/master/src/Graphics/Rendering/IGL/Buffer.hs#L114">See this</a>
and <a href="https://github.com/phaazon/igl/blob/master/src/Graphics/Rendering/IGL/GL.hs#L51">this</a>
for excerpts.</p>
<h2>Luminance</h2>
<p>In my desperate quest of providing a safer <strong>OpenGL</strong>’s API, I decided to create
a library from scratch called <a href="https://github.com/phaazon/luminance">luminance</a>.
That library is not really an <strong>OpenGL</strong> safe wrapper, but it’s very close to
being that.</p>
<p><code>luminance</code> provides the same objects than <strong>OpenGL</strong> does, but via a safer way
to create, access and use them. It’s an effort for providing safe abstractions
without destroying performance down and suited for graphics applications. It’s
not a 3D engine. It’s a rendering framework. There’s no <em>light</em>, <em>asset
managers</em> or that kind of features. It’s just a <em>tiny</em> and <em>simple</em> powerful
API.</p>
<h3>Example</h3>
<p><code>luminance</code> is still a huge <em>work in progress</em>. However, I can already show an
example. The following example opens a window  but doesn’t render anything.
Instead, it creates a buffer on the GPU and perform several simple operations
onto it.</p>
<pre><code class="language-haskell">-- Several imports.
import Control.Monad.IO.Class ( MonadIO(..) )
import Control.Monad.Trans.Resource -- from the resourcet package
import Data.Foldable ( traverse_ )
import Graphics.Luminance.Buffer
import Graphics.Luminance.RW
import Graphics.UI.GLFW -- from the GLFW-b package
import Prelude hiding ( init ) -- clash with GLFW-b’s init function

windowW,windowH :: Int
windowW = 800
windowH = 600

windowTitle :: String
windowTitle = &quot;Test&quot;

main :: IO ()
main = do
  init
  -- Initiate the OpenGL context with GLFW.
  windowHint (WindowHint'Resizable False)
  windowHint (WindowHint'ContextVersionMajor 3)
  windowHint (WindowHint'ContextVersionMinor 3)
  windowHint (WindowHint'OpenGLForwardCompat False)
  windowHint (WindowHint'OpenGLProfile OpenGLProfile'Core)
  window &lt;- createWindow windowW windowH windowTitle Nothing Nothing
  makeContextCurrent window
  -- Run our application, which needs a (MonadIO m,MonadResource m) =&gt; m
  -- we traverse_ so that we just terminate if we’ve failed to create the
  -- window.
  traverse_ (runResourceT . app) window
  terminate

-- GPU regions. For this example, we’ll just create two regions. One of floats
-- and the other of ints. We’re using read/write (RW) regions so that we can
-- send values to the GPU and read them back.
data MyRegions = MyRegions {
    floats :: Region RW Float
  , ints   :: Region RW Int
  }

-- Our logic.
app :: (MonadIO m,MonadResource m) =&gt; Window -&gt; m ()
app window = do
  -- We create a new buffer on the GPU, getting back regions of typed data
  -- inside of it. For that purpose, we provide a monadic type used to build
  -- regions through the 'newRegion' function.
  region &lt;- createBuffer $
    MyRegions
      &lt;$&gt; newRegion 10
      &lt;*&gt; newRegion 5
  clear (floats region) pi -- clear the floats region with pi
  clear (ints region) 10 -- clear the ints region with 10
  readWhole (floats region) &gt;&gt;= liftIO . print -- print the floats as an array
  readWhole (ints region) &gt;&gt;= liftIO . print -- print the ints as an array
  floats region `writeAt` 7 $ 42 -- write 42 at index=7 in the floats region
  floats region @? 7 &gt;&gt;= traverse_ (liftIO . print) -- safe getter (Maybe)
  floats region @! 7 &gt;&gt;= liftIO . print -- unsafe getter
  readWhole (floats region) &gt;&gt;= liftIO . print -- print the floats as an array
</code></pre>
<p>Those read/write regions could also have been made <em>read-only</em> or <em>write-only</em>.
For such regions, some functions can’t be called, and trying to do so will make
your compiler angry and throw errors at you.</p>
<p>Up to now, the buffers are created <em>persistently</em> and <em>coherently</em>. That might
cause issues with <strong>OpenGL</strong> <em>synchronization</em>, but I’ll wait for benchmarks
before changing that part. If benchmarking spots performance bottlenecks, I’ll
introduce more buffers and regions to deal with special cases.</p>
<p><code>luminance</code> doesn’t force you to use a specific windowing library. You can then
embed it into any kind of host libraries.</p>
<h1>What’s to come?</h1>
<p><a href="https://github.com/phaazon/luminance">luminance</a> is very young. At the moment
of writing this article, it’s only 26 commits old. I just wanted to present it
so that people know it exists will be released as soon as possible. The idea is
to provide a library that, if you use it, won’t create black screens because of
framebuffers incorrectness or buffers issues. It’ll ease debugging <strong>OpenGL</strong>
applications and prevent from making nasty mistakes.</p>
<p>I’ll keep posting about <code>luminance</code> as I get new features implemented.</p>
<p>As always, keep the vibe, and happy hacking!</p>


guid: e611bba8-b1f8-11ed-89a2-6ba13f605733
title: Ep. 1 - Al Gore Saves the World...Again!
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Saint Albert wrings more money out of the rising seas. Plus, the Conservative Millennial, Zo Rachel, and Jacob Airey come on the Panel of Deplorables to talk Russia, war with North Korea, and the latest in Trump hate.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/c3fc694b-8790-41d1-8492-1cce7895bf6c/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=c3fc694b-8790-41d1-8492-1cce7895bf6c&feed=6c2VScgo


guid: e62690f0-b1f8-11ed-89a2-335a2f343b7b
title: Ep. 2 - MacArthur Vindicated: N. Korea Can Nuke Us
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Tin-pot dictator Kim Jong Un's missile can now reach the United States; Pres. Trump promises to "handle it." Plus, Roaming Millennial, Zo Rachel, and Amanda Prestigiacomo join the Panel of Deplorables to talk Tomi, Beyonce professors, and why Americans don't want their doctors to kill them.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/03e013a7-620d-4666-9d16-644a84d19252/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=03e013a7-620d-4666-9d16-644a84d19252&feed=6c2VScgo


guid: e5ef3934-b1f8-11ed-89a2-4b185d17d7b2
title: Ep. 3 - Affirmative Action: Warm And Fuzzy Race Discrimination
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>The DOJ has affirmative action on the chopping block. Plus, the Dow hits record highs, Trump sidesteps environmental regs on the border, and NASA defends the planet. Louder With Crowder's Not Gay Jared, Roaming Millennial, and Zo Rachel join the Panel of Deplorables to discuss.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/30cf4ce1-ee38-4742-9f0a-ef63c1ea3765/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=30cf4ce1-ee38-4742-9f0a-ef63c1ea3765&feed=6c2VScgo


guid: e61c0eb4-b1f8-11ed-89a2-230fc80cc4a1
title: Ep. 4 - Goo-Goo Ga-ttaca: The Dawn of Designer Babies
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Scientists successfully edit the DNA of human embryos—cue the übermenschen. Plus, Roaming Millennial, Amanda Prestigiacomo, and Paul Bois join to talk Rosie O'Donnell's all-women political party, Stephen Miller's CNN smackdown, and President Zuckerberg. Finally, Michael answers your questions in the first-ever MK mailbag!</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/092d537a-c80b-470f-b742-4f876dd7e48e/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=092d537a-c80b-470f-b742-4f876dd7e48e&feed=6c2VScgo


guid: e603bb2a-b1f8-11ed-89a2-db8ecd7c6e05
title: Ep. 5 - The Revolution Will Be Televised: Trump TV Debuts
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Trump TV debuts! Michael explains why conservatives should celebrate. Plus, Antonia Okafor, Roaming Millennial, and Jacob Airey join the Panel of Deplorables to talk Pence 2020, the surge in black female gun ownership, and how to tell if your child is transgender.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/1ecfcbc4-a45a-4559-aa54-bf91090eeaed/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=1ecfcbc4-a45a-4559-aa54-bf91090eeaed&feed=6c2VScgo


guid: e5e32be4-b1f8-11ed-89a2-8b64f17f7bfa
title: Ep. 6 - Google's Church of PC Excommunicates A Heretic
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>The tech giant inclusively and tolerantly fired an engineer for contradicting the gospel of PC. Plus, Amanda Prestigiacomo, Paul Bois, and Jacob Airey join the Panel of Deplorables to discuss bureaucrats' climate change coup, the MSM's declaration of Trump's demise, and coddled guide dogs.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/fd8a6f04-cfbb-4521-a223-5fde22173b3c/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=fd8a6f04-cfbb-4521-a223-5fde22173b3c&feed=6c2VScgo


guid: e5ee4bbe-b1f8-11ed-89a2-675b367cd5d2
title: Ep. 7 - ‘Gender Fluid’ Toddler? Send Zer To Trans Camp!
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Is your 4-year-old ‘gender fluid’? Well now you can send him or her or zim or zer to transgender summer camp for kids in, where else?, the San Francisco Bay Area. Plus, Roaming Millennial, Amanda Prestigiacomo, and Jacob Airey join the Panel of Deplorables to discuss, speaking of the youths, social media depression among kids, President Trump’s tough talk on North Korea, and the urgent need for the government to regulate sex robots.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/e9d873a3-9db1-4022-a522-208810ed6218/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=e9d873a3-9db1-4022-a522-208810ed6218&feed=6c2VScgo


guid: e61993b4-b1f8-11ed-89a2-9f0a549e7e30
title: Ep. 8 - FLECCAS TALKS Why The Right Wins Online
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Right-wing YouTube star joins Michael in-studio to talk conservative Internet culture. Plus, Antonia Okafor and Zo Rachel join the Panel of Deplorables to discuss giant chicken Trump, gay seat belts, and 13 million year old apes.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/cc2d3306-7450-46c8-b38b-b092db9036b9/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=cc2d3306-7450-46c8-b38b-b092db9036b9&feed=6c2VScgo


guid: e61f914c-b1f8-11ed-89a2-a3b748180330
title: Ep. 9 - Unite The Right, Not The Wrong: Alt-Right's Bad Religion
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Michael explains why the Alt-Right goes so wrong. Plus, Lone Conservative’s Kassy Dillon, Roaming Millennial, and the Daily Wire’s Jacob Airey join the Panel of Deplorables to discuss neo-Nazis losing their website, Obama’s election collusion with Russia, and for some reason Colin Kaepernick.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/4d12695d-7335-4e05-8b03-c23eb083a235/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=4d12695d-7335-4e05-8b03-c23eb083a235&feed=6c2VScgo


guid: e60fd194-b1f8-11ed-89a2-9b93c2b72ea8
title: Ep. 10 - American Taliban Blow Up Biloxi Buddhas
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>College town commies topple Confederate statues. Plus, Amanda Prestigiacomo, Jacob Airey, and Paul Bois join the Panel of Deplorables to discuss the Trump Train smushing CNN, Kumbaya nonsense, and the Bubonic Plague.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/cf9166c1-417a-41e6-a719-9174e30d6b2d/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=cf9166c1-417a-41e6-a719-9174e30d6b2d&feed=6c2VScgo


guid: e613b386-b1f8-11ed-89a2-eb4bfa8e029b
title: Ep. 11 - "Unite The Right" Speaker James Allsup Talks Charlottesville
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Michael talks with <a href="https://soundcloud.com/tags/UniteTheRight">#UniteTheRight</a> with scheduled speaker James Allsup. Plus, Josh Yasmeh, Aaron Bandler, and Amanda Presitigiacomo join the Panel of Deplorables to discuss Kim Jong Un backing down, Iceland's Down Syndrome genocide, and the most liked tweet in history.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/6a36e6cf-40c8-45aa-b9ce-264954c83fa4/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=6a36e6cf-40c8-45aa-b9ce-264954c83fa4&feed=6c2VScgo


guid: e62892b0-b1f8-11ed-89a2-7b3c8fb02379
title: Ep. 12 - SAD! A Week In Review
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Michael recaps a particularly rough week for President Trump and America. Fleccas Talks, Zo Rachel, and Josh Yasmeh join the Panel of Deplorables to talk how TV is dead, Netflix is killing you, and books have the worst words. Plus, the Mailbag!</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/2b54c11e-a825-402f-a018-800e23f3e82f/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=2b54c11e-a825-402f-a018-800e23f3e82f&feed=6c2VScgo


guid: e6036c42-b1f8-11ed-89a2-fb0608b04ba3
title: Ep. 13 - ‘Peace Through Violence’: Antifa And The ‘Alt-Left’
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Violent left-wing Antifa thugs assault police and right-wingers in Boston as 40,000 protesters surrounded fifty or so free speech demonstrators. Plus, Antonia Okafor, Amanda Prestigiacomo, and Jacob Airey join the Panel of Deplorables to talk killer robots, anti-Trump stunts on campus, and President Trump’s disbanding of a global warming advisory panel.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/1a431bcd-7f08-4b9d-92ee-fe535e1cd6aa/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=1a431bcd-7f08-4b9d-92ee-fe535e1cd6aa&feed=6c2VScgo


guid: e601f5ec-b1f8-11ed-89a2-2ffac2ad5061
title: Ep. 14 - Safe-Space Foreclosures: Colleges Pay Price For Coddling
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Campus hotbeds of snowflake appeasement see enrollment collapse. Michael analyzing the costs of coddling students. Plus, Roaming Millennial, Allie Stuckey, and Antonia Okafor join the Panel of Deplorables to talk Trump's Afghanistan strategy, hostile work environments, and Steve Mnuchin's rich wife.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/64c104bb-90b3-40f4-a434-25018c74a27d/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=64c104bb-90b3-40f4-a434-25018c74a27d&feed=6c2VScgo


guid: e5ed4ff2-b1f8-11ed-89a2-5b5984f7b05b
title: Ep. 15 - Trump Rallies, MSM Eats Covfefe
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>After President Trump’s rally, the MSM is admitted to urgent care for an overdose of covfefe. Plus, Antonia Okafor, Amanda Prestigiacomo, and Jacob Airey join the Panel of Deplorables to discuss ESPN’s removing an Asian sports reporter because his name is Robert Lee, the possible pardon of Sheriff Joe Arpaio, and Hillary’s endless whining that the American people rejected her.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/59fcaf2c-7f74-4f7b-93cd-aefcb12f09c8/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=59fcaf2c-7f74-4f7b-93cd-aefcb12f09c8&feed=6c2VScgo


guid: e5e34192-b1f8-11ed-89a2-074393c269b6
title: Ep. 16 - What Is Western Civilization?
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Oxford classicist Spencer Klavan sits down in studio to discuss what the West is and is not. Then, Kassy Dillon and Fleccas Talks join the Panel of Deplorables to talk Fake News, Trump's thrashing Congress, and the health benefits of the death penalty.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/05bbd64a-c5db-4e47-addf-684f6c13df62/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=05bbd64a-c5db-4e47-addf-684f6c13df62&feed=6c2VScgo


guid: e5e82b12-b1f8-11ed-89a2-473e75e527e3
title: Ep. 17 - Hurricane Harvey: Trump Sidesteps MSM Tsunami
author: The Daily Wire
url: https://www.dailywire.com/show/the-michael-knowles-show
feedurl: https://feeds.simplecast.com/6c2VScgo
enclosure_type: audio/mpeg
content_mime_type: text/html
content:
<p>Trump's Harvey response avoids Bush's Katrina gaffes, and the mainstream media are furious. Plus, Roaming Millennial, Not Gay Jared, and Amanda Prestigiacomo join the Panel of Deplorables to talk Amazon's cutting Whole Foods prices, soon-to-be-deported "Dreamers," and how best to buy happiness.</p><p> </p><p>Learn more about your ad choices. Visit <a href="https://podcastchoices.com/adchoices">podcastchoices.com/adchoices</a></p>

enclosure_url:
https://claritaspod.com/measure/arttrk.com/p/24FDE/verifi.podscribe.com/rss/p/pfx.vpixl.com/2jSe3/prfx.byspotify.com/e/dts.podtrac.com/redirect.mp3/mgln.ai/e/121/injector.simplecastaudio.com/e626ec69-4c64-4e58-9a75-6b64d301dd52/episodes/88202336-6a63-45a7-86d1-c8ec8995fc51/audio/128/default.mp3?aid=rss_feed&awCollectionId=e626ec69-4c64-4e58-9a75-6b64d301dd52&awEpisodeId=88202336-6a63-45a7-86d1-c8ec8995fc51&feed=6c2VScgo


