Teach-rs
Teach-rs is a university course for computer science students, introducing the Rust Programming Language, and is available for anyone who wants to teach Rust.
Why? Have a look at our blog post introducing the course.
This repo will contain everything that's needed to organize the course: slides, exercises, tools, setup instructions and more.
While all the available material is user-ready, this repo is not yet as exhaustive as we'd like, so feedback and contributions are welcome! So is sponsorship; read more below or on our Sponsorship page.
Need help? Have questions? Say hi in our Matrix channel:
Intent
Teach-rs is meant to be a collection of teaching material about the Rust programming language for use in higher education (whether that be universities focused on theoretical science, educational institutes of applied science, or higher vocational education). More broadly, it is intended to be also useful in any form of formal or informal classroom education (e.g., community colleges).
It is not a course in rust, but rather meant to be used by teachers to create a course in Rust tailored to the needs of their students. Hence, the name: teach-rs. That means that we expect (and encourage) every course that is taught using teach-rs to make different selections of material.
The goal of this project is therefore for teach-r to consist of:
- Exercise material: exercise instructions and templates.
- A reference slide deck for use in lectures.
- Tools to help teachers easily make selections of material that make sure that essential parts are not skipped.
What is not in scope of teach-rs:
- The equivalent of a "book", i.e., reading material; good external resources exist both online and in book form. We do want to have a section on collecting all of those.
- Being usable as self-teaching aid. That is not to say that the material contained in this repository cannot be used to become self-taught in Rust, but that is not the primary mission.
- Solutions to exercises. Some of our exercises are open-ended and have multiple correct solutions. In line with being meant for use in higher education, students should reflect on their solutions themselves, discuss their work with their peers or receive feedback from their teacher/teaching assistants. The Rust compiler itself also lends itself well as a teaching aid: it will catch many mistakes and suggest improvements; i.e., various exercises may try to steer students into interaction with Rust compiler messages.
The material is free for any purpose (licensed under CC-BY-SA). It is highly appreciated that changes/improvements are contributed back to us, even if the license doesn't necessarily demand it.
Usage
The teacher's guide can be found here. Have a look at the ModMod Readme for instructions on how to render the content of a track.
Structure
The actual content can be found in the content
directory.
The content is structured in a tree of Tracks, Modules, Units, and Topics.
Tracks define a single course, which consists of one or more Modules, which again combines one or more Units, which again is a set of Topics.
Units roughly correspond to one lecture+tutorial (or at least that is the idea, but TODO), and consist of several Topics. Related Units are combined in a Module.
Topics are packages that cover a single topic, and include a small number of slides, some exercises, and an exercise descripion.
Topics can define their learning objectives, further reading material, and how they should be summarized in a Unit introduction.
Tracks, Modules, Units, and Topics and the files they refer to are described in the several TOML files in the content
directory.
ModMod combines the content into a structure that can be directly published to your students in a Git repo, for instance.
Pre-defined tracks
- Rust Language Introduction aims to introduce the basics to the Rust programming language, and to enable students to engineer their own applications and crates.
- Rust for the Web covers content that is needed to use Rust in web applications.
- Rust for Systems Programming contains more low-level topics, to teach systems programming using Rust.
- Scientific Rust is about using Rust in scientific programming.
- Full contains all available teach-rs content.
Note: although the outline of the tracks is mostly complete, the tracks may still contain TODOs. You're invited to contribute your own content to fix these!
High-level goals
Teach-rs aims to provide an open-source course, lectures, tutorials and exercises, that can be used by any higher education institution. Use one of the pre-defined tracks, or compose your own with the content we provide and your own.
- Provide a modular, resuable basis for live-taught Rust courses
- Provide students with practical, hands-on experience
- Provide students with background information of Rust features
- Provide students with ability to judge whether Rust fits a project
- Provide several specialized learning tracks that focus on different applications (e.g. systems, embedded, web)
- Enable teachers to contribute their material for others to use
Contributing
If you'd like to improve teach-rs, either by doing touchups, restructuring a module, or even adding a module, please refer to the contributing guidelines before you get started.
About the project
The project was created by Tweede golf, and has since moved to the Trifecta Tech Foundation.
Our sponsors
Founding sponsors




The project's initial sponsor is the Faculty of Informatics and Information Technologies (FIIT) of the Slovak University of Technology (STU) in Bratislava, Slovakia. FIIT's contribution has enabled us to lay the groundwork for the course. Tweede golf and Rust Edu have also contributed substantially to the creation of teach-rs.
In addition, one of our maintainers, @hdoordt, received a grant from the Rust Foundation.
Silver sponsors
And a big thank you to our Silver sponsors:
Support teach-rs
Contact us if youΒ΄re interested in financially supporting the maintenance and further development of the teach-rs resources. See trifectatech.org/support. You can also sponsor our work through GitHub sponsors.
Unit 1 - Introduction
Exercise 1.1.1: Setup Your Installation
In this file you'll find instructions on how to install the tools we'll use during the course.
All of these tools are available for Linux, macOS and Windows users. We'll need the tools to write and compile our Rust code, and allow for remote mentoring. Important: these instructions are to be followed at home, before the start of the first tutorial. If you have any problems with installation, contact the lecturers! We won't be addressing installation problems during the first tutorial.
Rust and Cargo
First we'll need rustc
, the standard Rust compiler.
rustc
is generally not invoked directly, but through cargo
, the Rust package manager.
rustup
takes care of installing rustc
and cargo
.
This part is easy: go to https://rustup.rs and follow the instructions. Please make sure you're installing the latest default toolchain. Once done, run
rustc -V && cargo -V
The output should be something like this:
rustc 1.67.1 (d5a82bbd2 2023-02-07)
cargo 1.67.1 (8ecd4f20a 2023-01-10)
Using Rustup, you can install Rust toolchains and components. More info:
Rustfmt and Clippy
To avoid discussions, Rust provides its own formatting tool, Rustfmt. We'll also be using Clippy, a collection of lints to analyze your code, that catches common mistakes for you. You'll notice that Rusts Clippy can be a very helpful companion. Both Rustfmt and Clippy are installed by Rustup by default.
To run Rustfmt on your project, execute:
cargo fmt
To run clippy:
cargo clippy
More info:
Visual Studio Code
During the course, we will use Visual Studio Code (vscode) to write code in. Of course, you're free to use your favorite editor, but if you encounter problems, you can't rely on support from us. Also, we'll use vscode to allow for remote collaboration and mentoring during tutorial sessions.
You can find the installation instructions here: https://code.visualstudio.com/.
We will install some plugins as well. The first one is Rust-Analyzer. Installation instructions can be found here https://marketplace.visualstudio.com/items?itemName=rust-lang.rust-analyzer. Rust-Analyzer provides a lot of help during development and in indispensable when getting started with Rust.
Another plugin we'll install is Live Share. We will use the plugin to share screens and provide help during remote tutorial sessions. The extension pack also contains the Live Share Audio plugin, which allows for audio communication during share sessions. Installation instructions can be found here: https://marketplace.visualstudio.com/items?itemName=MS-vsliveshare.vsliveshare
The last plugin we'll use is CodeLLDB. This plugin enables debugging Rust code from within vscode. You can find instructions here: https://marketplace.visualstudio.com/items?itemName=vadimcn.vscode-lldb.
More info:
Git
We will use Git as version control tool. If you haven't installed Git already, you can find instructions here: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git. If you're new to Git, you'll also appreciate GitHubs intro to Git https://docs.github.com/en/get-started/using-git/about-git and the Git intro with vscode, which you can find here: https://www.youtube.com/watch?v=i_23KUAEtUM.
More info: https://www.youtube.com/playlist?list=PLg7s6cbtAD15G8lNyoaYDuKZSKyJrgwB-
Course code
Now that everything is installed, you can clone the source code repository. The repository can be found here: https://github.com/tweedegolf/teach-rs.
Instructions on cloning the repository can be found here: https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls
Trying it out
Now that you've got the code on your machine, navigate to it using your favorite terminal and run:
cd exercises/1-course-introduction/1-introduction/1-setup-your-installation
cargo run
This command may take a while to run the first time, as Cargo will first fetch the crate index from the registry.
It will compile and run the intro
package, which you can find in exercises/1-course-introduction/1-introduction/1-setup-your-installation
.
If everything goes well, you should see some output:
Compiling intro v0.1.0 ([REDACTED]/exercises/1-course-introduction/1-introduction/1-setup-your-installation)
Finished dev [unoptimized + debuginfo] target(s) in 0.11s
Running `target/debug/intro`
π¦ Hello, world! π¦
You've successfully compiled and run your first Rust project!
If Rust-Analyzer is set up correctly, you can also click the 'βΆοΈ Run'-button that is shown in exercises/1-course-introduction/1-introduction/1-setup-your-installation/src/main.rs
.
With CodeLLDB installed correctly, you can also start a debug session by clicking 'Debug', right next to the 'βΆοΈ Run'-button.
Play a little with setting breakpoints by clicking on a line number, making a red circle appear and stepping over/into/out of functions using the controls.
You can view variable values by hovering over them while execution is paused, or by expanding the 'Local' view under 'Variables' in the left panel during a debug session.
Unit 2.1 - Basic Syntax
Exercise 2.1.1: Basic Syntax
Open exercises/2-foundations-of-rust/1-basic-syntax/1-basic-syntax
in your editor. This folder contains a number of exercises with which you can practise basic Rust syntax.
While inside the exercises/2-foundations-of-rust/1-basic-syntax/1-basic-syntax
folder, to get started, run:
cargo run --bin 01
This will try to compile exercise 1. Try and get the example to run, and continue on with the next exercise by replacing the number of the exercise in the cargo run command.
Some exercises contain unit tests. To run the test in src/bin/01.rs
, run
cargo test --bin 01
Make sure all tests pass!
Unit 2.2 - Ownership and References
Exercise 2.2.1: Move Semantics
This exercise is adapted from the move semantics exercise from Rustlings
While inside the exercises/2-foundations-of-rust/2-ownership-and-references/1-move-semantics
folder, to get started, run:
cargo run --bin 01
This will try to compile exercise 1. Try and get the example to run, and continue on with the next exercise by replacing the number of the exercise in the cargo run command.
Some exercises contain unit tests. To run the test in src/bin/01.rs
, run
cargo test --bin 01
Make sure all tests pass!
01.rs
should compile as is, but you'll have to make sure the others compile as well. For some exercises, instructions are included as doc comments at the top of the file. Make sure to adhere to them.
Exercise 2.2.2: Borrowing
Fix the two examples in the exercises/2-foundations-of-rust/2-ownership-and-references/2-borrowing
crate! Don't forget you
can run individual binaries by using cargo run --bin 01
in that directory!
Make sure to follow the instructions that are in the comments!
Unit 2.3 - Advanced Syntax
Exercise 2.3.1: Error propagation
Follow the instructions in the comments of exercises/2-foundations-of-rust/3-advanced-syntax/1-error-propagation/src/main.rs
!
Exercise 2.3.2: Error handling
Follow the instructions in the comments of exercises/2-foundations-of-rust/3-advanced-syntax/2-error-handling/src/main.rs
!
Exercise 2.3.3: Slices
Follow the instructions in the comments of exercises/2-foundations-of-rust/3-advanced-syntax/3-slices/src/main.rs
!
Don't take too much time on the extra assignment, instead come back later once
you've done the rest of the exercises.
Exercise 2.3.4: Ring Buffer
This is a bonus exercise! Follow the instructions in the comments of
exercises/2-foundations-of-rust/3-advanced-syntax/4-ring-buffer/src/main.rs
!
Exercise 2.3.5: Boxed Data
Follow the instructions in the comments of exercises/2-foundations-of-rust/3-advanced-syntax/5-boxed-data/src/main.rs
!
Unit 2.4 - Traits and Generics
Exercise 2.4.1: Local Storage Vec
In this exercise, we'll create a type called LocalStorageVec
, which is generic list of items that resides either on the stack or the heap, depending on its size. If its size is small enough for items to be put on the stack, the LocalStorageVec
buffer is backed by an array. LocalStorageVec
is not only generic over the type (T
) of items in the list, but also by the size (N
) of this stack-located array using a relatively new feature called "const generics". Once the LocalStorageVec
contains more items than fit in the array, a heap based Vec
is allocated as space for the items to reside in.
Within this exercise, the objectives are annotated with a number of stars (β), indicating the difficulty. You are likely not to be able to finish all exercises during the tutorial session
Questions
- When is such a data structure more efficient than a standard
Vec
? - What are the downsides, compared to just using a
Vec
?
Open the exercises/2-foundations-of-rust/4-traits-and-generics/1-local-storage-vec
crate. It contains a src/lib.rs
file, meaning this crate is a library. lib.rs
contains a number of tests, which can be run by calling cargo test
. Don't worry if they don't pass or even compile right now: it's your job to fix that in this exercise. Most of the tests are commented out right now, to enable a step-by-step approach. Before you begin, have a look at the code and the comments in there, they contain various helpful clues.
2.4.1.A Defining the type β
Currently, the LocalStorageVec
enum
is incomplete. Give it two variants: Stack
and Heap
. Stack
contains two named fields, buf
and len
. buf
will be the array with a capacity to hold N
items of type T
; len
is a field of type usize
that will denote the amount of items actually stored. The Heap
variant has an unnamed field containing a Vec<T>
. If you've defined the LocalStorageVec
variants correctly, running cargo test
should output something like
running 1 test
test test::it_compiles ... ignored, This test is just to validate the definition of `LocalStorageVec`. If it compiles, all is OK
test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s
This test does (and should) not run, but is just there for checking your variant definition.
Hint 1
You may be able to reverse-engineer the `LocalStorageVec` definition using the code of the `it_compiles` test case.Hint 2 (If you got stuck, but try to resist me for a while)
Below definition works. Read the code comments and make sure you understand what's going on.
#![allow(unused)] fn main() { // Define an enum `LocalStorageVec` that is generic over // type `T` and a constant `N` of type `usize` pub enum LocalStorageVec<T, const N: usize> { // Define a struct-like variant called `Stack` containing two named fields: // - `buf` is an array with elements of `T` of size `N` // - `len` is a field of type `usize` Stack { buf: [T; N], len: usize }, // Define a tuple-like variant called `Heap`, containing a single field // of type `Vec<T>`, which is a heap-based growable, contiguous list of `T` Heap(Vec<T>), } }
2.4.1.B impl
-ing From<Vec<T>>
β
Uncomment the test it_from_vecs
, and add an implementation for From<Vec<T>>
to LocalStorageVec<T>
. To do so, copy the following code in your lib.rs
file and replace the todo!
macro invocation with your code that creates a heap-based LocalStorageVec
containing the passed Vec<T>
.
#![allow(unused)] fn main() { impl<T, const N: usize> From<Vec<T>> for LocalStorageVec<T, N> { fn from(v: Vec<T>) -> Self { todo!("Implement me"); } } }
Question
- How would you pronounce the first line of the code you just copied in English?*
Run cargo test
to validate your implementation.
2.4.1.C impl LocalStorageVec
ββ
To make the LocalStorageVec
more useful, we'll add more methods to it.
Create an impl
-block for LocalStorageVec
.
Don't forget to declare and provide the generic parameters.
For now, to make implementations easier, we will add a bound T
, requiring that it implements Copy
and Default
.
First off, uncomment the test called it_constructs
.
Make it compile and pass by creating a associated function called new
on LocalStorageVec
that creates a new, empty LocalStorageVec
instance without heap allocation.
The next methods we'll implement are len
, push
, pop
, insert
, remove
and clear
:
len
returns the length of theLocalStorageVec
push
appends an item to the end of theLocalStorageVec
and increments its length. Possibly moves the contents to the heap if they no longer fit on the stack.pop
removes an item from the end of theLocalStorageVec
, optionally returns it and decrements its length. If the length is 0,pop
returnsNone
insert
inserts an item at the given index and increments the length of theLocalStorageVec
remove
removes an item at the given index and returns it.clear
resets the length of theLocalStorageVec
to 0.
Uncomment the corresponding test cases and make them compile and pass. Be sure to have a look at the methods provided for slices [T]
and Vec<T>
Specifically, [T]::copy_within
and Vec::extend_from_slice
can be of use.
2.4.1.E Iterator
and IntoIterator
ββ
Our LocalStorageVec
can be used in the real world now, but we still shouldn't be satisfied. There are various traits in the standard library that we can implement for our LocalStorageVec
that would make users of our crate happy.
First off, we will implement the IntoIterator
and Iterator
traits. Go ahead and uncomment the it_iters
test case. Let's define a new type:
#![allow(unused)] fn main() { pub struct LocalStorageVecIter<T, const N: usize> { vec: LocalStorageVec<T, N>, counter: usize, } }
This is the type we'll implement the Iterator
trait on. You'll need to specify the item this Iterator
implementation yields, as well as an implementation for Iterator::next
, which yields the next item. You'll be able to make this easier by bounding T
to Default
when implementing the Iterator
trait, as then you can use the std::mem::take
function to take an item from the LocalStorageVec
and replace it with the default value for T
.
Take a look at the list of methods under the 'provided methods' section. In there, lots of useful methods that come free with the implementation of the Iterator
trait are defined, and implemented in terms of the next
method. Knowing in the back of your head what methods there are, greatly helps in improving your efficiency in programming with Rust. Which of the provided methods can you override in order to make the implementation of LocalStorageVecIter
more efficient, given that we can access the fields and methods of LocalStorageVec
?
Now to instantiate a LocalStorageVecIter
, implement the [IntoIter
] trait for it, in such a way that calling into_iter
yields a LocalStorageVecIter
.
2.4.1.F Index
ββ
To allow users of the LocalStorageVec
to read items or slices from its buffer, we can implement the Index
trait. This trait is generic over the type of the item used for indexing. In order to make our LocalStorageVec
versatile, we should implement:
Index<usize>
, allowing us to get a single item by callingvec[1]
;Index<RangeTo<usize>>
, allowing us to get the firstn
items (excluding itemn
) by callingvec[..n]
;Index<RangeFrom<usize>>
, allowing us to get the lastn
items by callingvec[n..]
;Index<Range<usize>>
, allowing us to get the items betweenn
andm
items (excluding itemm
) by callingvec[n..m]
;
Each of these implementations can be implemented in terms of the as_ref
implementation, as slices [T]
all support indexing by the previous types. That is, [T]
also implements Index
for those types. Uncomment the it_indexes
test case and run cargo test
in order to validate your implementation.
2.4.1.G Removing bounds ββ
When we implemented the borrowing Iterator
, we saw that it's possible to define methods in separate impl
blocks with different type bounds. Some of the functionality you wrote used the assumption that T
is both Copy
and Default
. However, this means that each of those methods are only defined for LocalStorageVec
s containing items of type T
that in fact do implement Copy
and Default
, which is not ideal. How many methods can you rewrite having one or both of these bounds removed?
2.4.1.H Borrowing Iterator
βββ
We've already got an iterator for LocalStorageVec
, though it has the limitation that in order to construct it, the LocalStorageVec
needs to be consumed. What if we only want to iterate over the items, and not consume them? We will need another iterator type, one that contains an immutable reference to the LocalStorageVec
and that will thus need a lifetime annotation. Add a method called iter
to LocalStorageVec
that takes a shared &self
reference, and instantiates the borrowing iterator. Implement the Iterator
trait with the appropriate Item
reference type for your borrowing iterator. To validate your code, uncomment and run the it_borrowing_iters
test case.
Note that this time, the test won't compile if you require the items of LocalStorageVec
be Copy
! That means you'll have to define LocalStorageVec::iter
in a new impl
block that does not put this bound on T
:
#![allow(unused)] fn main() { impl<T: Default + Copy, const N: usize> LocalStorageVec<T, N> { // Methods you've implemented so far } impl<T: const N: usize> LocalStorageVec<T, N> { pub fn iter(&self) -> /* TODO */ } }
Defining methods in separate impl
blocks means some methods are not available for certain instances of the generic type. In our case, the new
method is only available for LocalStorageVec
s containing items of type T
that implement both Copy
and Default
, but iter
is available for all LocalStorageVec
s.
2.4.1.I Generic Index
ββββ
You've probably duplicated a lot of code in exercise 2.4.1.F. We can reduce the boilerplate by defining an empty trait:
#![allow(unused)] fn main() { trait LocalStorageVecIndex {} }
First, implement this trait for usize
, RangeTo<usize>
, RangeFrom<usize>
, and Range<usize>
.
Next, replace the multiple implementations of Index
with a single implementation. In English:
"For each type T
, I
and constant N
of type usize
,
implement Index<I>
for LocalStorageVec<T, N>
,
where I
implements LocalStorageVecIndex
and [T]
implements Index<I>
"
If you've done this correctly, it_indexes
should again compile and pass.
2.4.1.J Deref
and DerefMut
ββββ
The next trait that makes our LocalStorageVec
more flexible in use are Deref
and DerefMut
that utilize the 'deref coercion' feature of Rust to allow types to be treated as if they were some type they look like.
That would allow us to use any method that is defined on [T]
by calling them on a LocalStorageVec
.
Before continuing, read the section 'Treating a Type Like a Reference by Implementing the Deref Trait' from The Rust Programming Language (TRPL).
Don't confuse deref coercion with any kind of inheritance! Using Deref
and DerefMut
for inheritance is frowned upon in Rust.
Below, an implementation of Deref
and DerefMut
is provided in terms of the AsRef
and AsMut
implementations. Notice the specific way in which as_ref
and as_mut
are called.
#![allow(unused)] fn main() { impl<T, const N: usize> Deref for LocalStorageVec<T, N> { type Target = [T]; fn deref(&self) -> &Self::Target { <Self as AsRef<[T]>>::as_ref(self) } } impl<T, const N: usize> DerefMut for LocalStorageVec<T, N> { fn deref_mut(&mut self) -> &mut Self::Target { <Self as AsMut<[T]>>::as_mut(self) } } }
Question
- Replacing the implementation of
deref
withself.as_ref()
results in a stack overflow when running an unoptimized version. Why? (Hint: deref coercion)
Unit 2.5 - Closures and Dynamic dispatch
Exercise 2.5.1: Config Reader
In this exercise, you'll work with dynamic dispatch to deserialize with serde_json
or serde_yaml
, depending on the file extension. The starter code is in exercises/2-foundations-of-rust/5-closures-and-dynamic-dispatch/1-config-reader
. Fix the todo's in there.
To run the program, you'll need to pass the file to deserialize to the binary, not to Cargo. To do this, run
cargo run -- <FILE_PATH>
Deserializing both config.json
and config.yml
should result in the Config
being printed correctly.
Unit 2.6 - Interior mutability
There are no exercises for this unit
Unit 3.1 - Crate Engineering
Exercise 3.1.1: My Serde App
This exercise is adapted from the serde_lifetimes exercise by Ferrous Systems
Open exercises/3-crate-engineering/1-crate-engineering/1-my-serde-app/src/main.rs
. In there, you'll find some Rust code we will do this exercise with.
We used todo!()
macros to mark places where you should put code to make the program run. Look at the serde_json
api for help.
Hint
Serde comes with two traits: `Serializable` and `Deserializable`. These traits can be `derive` d for your `struct` or `enum` types. Other `serde-*` crates use these traits to convert our data type from and to corresponding representation (`serde-json` to JSON, `serde-yaml` to YAML, etc.).How come
main
returns ananyhow::Result<()>
? By havingmain
return a result, we can bubble errors up all the way to runtime. You can find more information about it in Rust By Example. Theanyhow::Result
is a more flexible type ofResult
, which allows for easy conversion of error types.
What is that
r#"...
thing?
r
in front of a string literal means it's a "raw" string. Escape sequences (\n
,\"
, etc.) don't work, and thus they are very convenient for things like regular expressions, JSON literals, etc.Optionally
r
can be followed by one or more symbols (like#
in our case), and then your string ends when there's a closing double quote followed by the same number of the same symbols. This is great for cases when you want to have double quotes inside your string literal. For our exampler#" ... "#
works great for JSON. In rare cases you'd want to put two or more pound signs. Like, when you store CSS color values in your JSON strings:
#![allow(unused)] fn main() { // here `"#` would not terminate the string r##" { "color": "#ff00ff" } "## }
Exercise 3.1.2: Quizzer
In this exercise, you will create a Rust crate that adheres to the guidelines that were pointed out during the lecture. Additionally, you will add and use dependencies, create unit tests, and create some documentation. You can view this exercise as a stepping stone to the final project.
This exercise should be done in groups of 2 people
3.1.2.A Setting up β
Create a new project using cargo new --name quizzer
. Make sure it acts as both a binary and a library. That means there will be both a src/lib.rs
and a src/bin/quizzer/main.rs
file in your crate, where quizzer
is the name of the binary:
$ tree
.
βββ Cargo.toml
βββ quiz.json
βββ src
βββ bin
βΒ Β βββ quizzer
βΒ Β βββ main.rs
βββ lib.rs
Add the following dependencies to your Cargo.toml
file. Below items contain links to their page on lib.rs. Make sure you get a general idea of what these crates are for and how they can be used. Don't dive too deep just yet.
anyhow
1.0clap
4.0 Also, skim over https://docs.rs/clap/latest/clap/_derive/_tutorial/index.htmlserde-json
1.0serde
1.0
Your Cargo.toml
should look like this:
[package]
name = "quizzer"
version = "0.1.0"
edition = "2021"
### See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
anyhow = "1.0.66"
clap = { version = "4.0.18", features = ["derive"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0.87"
For clap
and serde
, the non-standard derive
feature of each these crates is enabled. For clap
, it allows us to derive the Parser
trait, which greatly simplifies creating a CLI. The derive
feaure from serde
allows us to derive the Serialize
and Deserialize
traits on any struct we wish to serialize or deserialize using serde
and its backends, in our case serde_json
.
3.1.2.B Quizzer βββ
This exercise is about both design and finding information. You'll have to figure out a model to represent your quiz questions, as well as a means to store them into a JSON file, and load them yourself. Also, you will have to find out how to parse the program arguments.
We will use the project we just set up to write a quiz game creator and player. You may add other dependencies as needed. It has the following functional requirements:
- It runs as a command-line tool in your terminal.
- It has two modes: question-entering mode and quiz mode. The mode is selected with a subcommand, passed as the first argument to the program.
- Question-entering mode: Allows for entering multiple-choice quiz questions, with 4 possible answers each, exactly 1 of them being correct. The questions are stored on disk as a JSON file.
- Quiz mode: Loads stored questions from the JSON file, presents the questions one-by-one to the player, reads and verifies the player input, and presents the score at the end of the game.
- Errors are correctly handled, i.e. your application does not panic if it encounters any unexpected situation. Use
anywhow
and the question-mark (?
) operator to make error-bubbling concise. You can read about the?
-operator here: https://doc.rust-lang.org/reference/expressions/operator-expr.html#the-question-mark-operator - Logic concerning creating, storing, and loading quiz questions is defined in the library part of your crate.
- Functionality regarding user input (arg parsing, reading from stdin) is defined in the application code, not in your library.
- Logical units of your crate are divided up into modules.
Before you start coding, make sure you've listed all open questions and found answers to them. You're also encouraged to draw a simple diagram of the module structure of your application, annotating each module with its responsibilities.
Exercise 3.1.3: BSN
The BSN (Burgerservicennummer) is a Dutch personal identification number that somewhat resembles the US Social Security Number in its use. The BSN is a number that adheres to some rules. In this exercise, we will create a Rust type that guarantees that it represents a valid BSN.
3.1.3.A Newtype ββ
In this part we will implement the BSN number validation, as well as a fallible constructor.
A BSN is valid if and only if it matches the following criteria:
- It consists of 8 or 9 digits
- It passes a variant of the 11 check (elfproef (Dutch)):
For 8-digit BSNs, we concatenate a 0
to the end. The digits of the number are labeled as ABCDEFGHI
.
For example: for BSN 123456789
, A = 1
, B = 2
, C = 3
, and so forth until I
.
Then, (9 Γ A) + (8 Γ B) + (7 Γ C) + (6 Γ D) + (5 Γ E) + (4 Γ F) + (3 Γ G) + (2 Γ H) + (-1 Γ I)
must be a multiple of 11
Open exercises/3-crate-engineering/1-crate-engineering/3-bsn
in your editor. You'll find the scaffolding code there, along with two files:
valid_bsns.in
containing a list of valid BSNsinvalid_bsns.in
containing a list of invalid BSNs.
In src/lib.rs
, implement Bsn::validate
to make the test_validation
test case pass.
Implement Bsn::try_from_string
as well.
To try just the test_validation
test case, run:
cargo test -- test_validation
3.1.3.B Visitor with Serde βββ
Next up is implementing the serde::Serialize
and serde::Deserialize
traits, to support serialization and deserialization of Bsn
s.
In this case, simply deriving those traits won't suffice, as we want to represent the BSN
as a string after serialization.
We also want to deserialize strings directly into Bsn
s, while still upholding the guarantee that an instantiated Bsn
represents a valid BSN.
Therefore, you have to incorporate Bsn::validate
into the implementation of the deserialization visitor.
More information on implementing the traits:
serde::Serialize
: https://serde.rs/impl-serialize.htmlserde::Deserialize
: https://serde.rs/impl-deserialize.html
If everything works out, all tests should pass.
Exercise 3.1.4: 3D Printer
An imaginary 3D printer uses filament to create all kinds of things. Its states can be represented with the following state diagram:
βββββββββββββββββββ
β β
β β Reset
β Idle βββββββββββββββββββββββββββββββ
ββββββββββΊβ β β
β β β β
β β β β
β ββββββββββ¬βββββββββ β
β β β
β β β
β β Start β
β β β
β βΌ β
β βββββββββββββββββββ ββββββββββ΄βββββββββ
β β β β β
β β β Out of filament β β
Product β β Printing ββββββββββββββββββββΊ β Error β
retrievedβ β β β β
β β β β β
β β β β β
β ββββββββββ¬βββββββββ βββββββββββββββββββ
β β
β β Product ready
β β
β βΌ
β βββββββββββββββββββ
β β β
β β β
β β Product Ready β
βββββββββββ€ β
β β
β β
βββββββββββββββββββ
The printer boots in Idle state. Once a job is started, the printer enters the Printing state. In printing state, it keeps on printing the product until either it is ready or the printer is out of filament. If the printer is out of filament, the printer goes into Error state, which it can only come out of upon device reset. If the product is ready, the printer goes to Product Ready state, and once the user retrieves the product, the printer goes back to Idle.
The printer can be represented in Rust using the typestate pattern as described during the lecture. This allows you to write a simple 3D printer driver. In exercises/3-crate-engineering/1-crate-engineering/4-3d-printer/src/lib.rs
, a Printer3D
struct is instantiated. Add methods corresponding to each of the types, that simulate the state transitions by printing the state. A method simulating checking if the printer is out of filament is provided.
Of course, to make the printer more realistic, you can add more states and transitions.
Exercise 3.1.5: FizzBuzz
In this exercise, you will practise writing a unit test, and use Rusts benchmarking functionality to help you optimize a FizzBuzz app. You will need cargo-criterion
, a tool that runs benchmarks and creates nice reports. You can install it by running
cargo install cargo-criterion --version=1.1.0
3.1.5.A Testing Fizz Buzz β
Open exercises/3-crate-engineering/1-crate-engineering/5-fizzbuzz/src/lib.rs
. Create a unit test that verifies the correctness of the fizz_buzz
function. You can use the include_str
macro to include exercises/3-crate-engineering/1-crate-engineering/5-fizzbuzz/fizzbuzz.out
as a &str
into your binary. Each line of fizzbuzz.out
contains the expected output of the fizz_buzz
function given the line number as input. You can run the test with
cargo test
By default, Rusts test harness captures all output and discards it, If you like to debug your test code using print statements, you can run
cargo test -- --nocapture
to prevent the harness from capturing output.
3.1.5.B Benchmarking Fizz Buzz ββ
You'll probably have noticed the fizz_buzz
implementation is not very optimized. We will use criterion
to help us benchmark fizz_buzz
. To run a benchmark, run the following command when in the exercises/3-crate-engineering/1-crate-engineering/5-fizzbuzz/
directory:
cargo criterion
This command will run the benchmarks, and report some statistics to your terminal. It also generates HTML reports including graphs that you can find under target/criterion/reports
. For instance, target/criterion/reports/index.html
is a summary of all benchmark. Open it with your browser and have a look.
Your job is to do some optimization of the fizz_buzz
function, and use cargo-criterion
to measure the impact of your changes. Don't be afraid to change the signature of fizz_buzz
, if, for instance, you want to minimize the number of allocations done by this function. However, make sure that the function is able to correctly produce the output. How fast can you FizzBuzz?
Unit 4.1 - Introduction to Multitasking
There are no exercises for this unit
Unit 4.2 - Parallel Multitasking
Exercise 4.2.1: TF-IDF
Follow the instructions in the comments of exercises/4-multitasking/2-parallel-multitasking/1-tf-idf/src/main.rs
!
Exercise 4.2.2: Mutex
The basic mutex performs a spin-loop while waiting to take the lock. That is terribly inefficient. Luckily, your operating system is able to wait until the lock becomes available, and will just put the thread to sleep in the meantime.
This functionality is exposed in the atomic_wait crate. The section on implementing a mutex from "Rust Atomics and Locks" explains how to use it.
- change the
AtomicBool
for aAtomicU32
- implement
lock
. Be careful about spurious wakes: afterwait
returns, you must stil check the condition - implement unlocking (
Drop for MutexGuard<T>
usingwake_one
.
The linked chapter goes on to further optimize the mutex. This is technically out of scope for this course, but we won't stop you if you try (and will still try to help if you get stuck)!
Unit 4.3 - Asynchronous Multitasking
Exercise 4.3.1: Async Channels
Channels are a very useful way to communicate between threads and async
tasks. They allow for decoupling your application into many tasks. You'll see how that can come in nicely in exercise E.2. In this exercise, you'll implement two variants: a oneshot channel and a multi-producer-single-consumer (MPSC) channel. If you're up for a challenge, you can write a broadcast channel as well.
4.3.1.A MPSC channel ββ
A multi-producer-single-consumer (MPSC) channel is a channel that allows for multiple Sender
s to send many messages to a single Receiver
.
Open exercises/4-multitasking/3-asynchronous-multitasking/1-async-channels
in your editor. You'll find the scaffolding code there. For part A, you'll work in src/mpsc.rs
. Fix the todo!
s in that file in order to make the test pass. To test, run:
cargo test -- mpsc
If your tests are stuck, probably either your implementation does not use the Waker
correctly, or it returns Poll::Pending
where it shouldn't.
4.3.1.B Oneshot channel βββ
A oneshot is a channel that allows for one Sender
to send exactly one message to a single Receiver
.
For part B, you'll work in src/broadcast.rs
. This time, you'll have to do more yourself. Intended behavior:
Receiver
implementsFuture
. It returnsPoll::Ready(Ok(T))
ifinner.data
isSome(T)
,Poll::Pending
ifinner.data
isNone
, andPoll::Ready(Err(Error::SenderDropped))
if theSender
was dropped.Receiver::poll
replacesinner.waker
with the one from theContext
.Sender
consumesself
on send, allowing the it to be used no more than once. Sending setsinner.data
toSome(T)
. It returnsErr(Error::ReceiverDropped(T))
if theReceiver
was dropped before sending.Sender::send
wakesinner.waker
after putting the data ininner.data
- Once the
Sender
is dropped, it marks itself dropped withinner
- Once the
Receiver
is dropped, it marks itself dropped withinner
- Upon succesfully sending the message, the consumed
Sender
is not marked as dropped. Insteadstd::mem::forget
is used to avoid running the destructor.
To test, run:
cargo test -- broadcast
4.3.1.C Broadcast channel (bonus) ββββ
A Broadcast channel is a channel that supports multiple senders and receivers. Each message that is sent by any of the senders, is received by every receiver. Therefore, the implemenentation has to hold on to messages until they have been sent to every receiver that has not yet been dropped. This furthermore implies that the message shoud be cloned upon broadcasting.
For this bonus exercise, we provide no scaffolding. Take your inspiration from the mpsc
and oneshot
modules, and implement a broadcast
module yourself.
Exercise 4.3.2: Async Chat
In this exercise, you'll write a simple chat server and client based on Tokio. Open exercises/4-multitasking/3-asynchronous-multitasking/2-async-chat
in your editor. The project contains a lib.rs
file, in which a type Message
resides. This Message
defines the data the chat server and clients use to communicate.
4.3.2.A Server βββ
The chat server, which resides in src/bin/server.rs
listens for incoming TCP connections on port 8000, and spawns two tasks (futures):
handle_incoming
: reads lines coming in from the TCP connection. It reads the username the client provides, and broadcasts incomingMessages
, possibly after some modification.handle_outgoing
: sends messages that were broadcasted by thehandle_incoming
tasks to the client over TCP.
Both handle_incoming
and handle_outgoing
contain a number to todo
s. Fix them.
To start the server, run
cargo run --bin server
4.3.2.B Client ββ
The chat client, residing in src/bin/client.rs
contains some todo's as well. Fix them to allow for registration and sending Message
s to the server.
To start the client, run
cargo run --bin client
If everything works well, you should be able to run multiple clients and see messages sent from each client in every other.
Unit 5.1 - Rust for Web
Exercise 5.1.1: Pastebin
This exercise is about writing a simple pastebin web server. Like the quizzer app, you will need to set up the project yourself. This webserver will be powered by axum
.
- Data is kept in memory. Bonus if you use a database or
sqlite
, but first make the app function properly without. - Expose a route to which a POST request can be sent, that accepts some plain text, and stores it along with a freshly generated UUID. The UUID is sent in the response. You can use the
uuid
crate to generate UUIDs. - Expose a route to which a GET request can be sent, that accepts a UUID and returns the plain text corresponding to the UUID, or a 404 error if it doesn't exist.
- Expose a route to which a DELETE request can be sent, that accepts a UUID and deletes the plain text corresonding to that UUID.
Unit 6 - Rust for Systems Programming
Exercise 6.1: Linked List
Follow the instructions in the comments of exercises/6-rust-for-systems-programming/1-foreign-function-interface/1-linked-list/src/bin/unsafe.rs
!
Exercise 6.2: execve
Follow the instructions in exercises/F/2-execve/README.md
and implement in exercises/F/2-execve/src/main.rs
!
Exercise 6.3: Tagges union
Follow the instructions in the comments of exercises/6-rust-for-systems-programming/1-foreign-function-interface/3-tagges-union/src/main.rs
!
Exercise 6.4: CRC in C
Use a CRC checksum function written in C in a Rust program
prerequisites
- A C compiler
Steps
-
Add the
cc
build dependency, by adding toCrate.toml
the lines:[build-dependencies] cc = "1.0"
-
Create
build.rs
with contentsextern crate cc; fn main() { println!("cargo:rerun-if-changed=crc32.h"); println!("cargo:rerun-if-changed=crc32.c"); cc::Build::new().file("crc32.c").compile("crc32.a"); }
This will find your c code, compile it, and link it into the executable rust produces
-
In
main.rs
, define an extern (fill in the argument and return types)#![allow(unused)] fn main() { extern "C" { fn CRC32( ... ) -> ...; // hint: https://doc.rust-lang.org/std/os/raw } }
-
Now, create a rust wrapper that calls the extern function
#![allow(unused)] fn main() { fn crc32( ... ) -> ... { ... // (hints: `unsafe`, `.as_ptr()`, `.len()`) } }
-
Call our wrapper on some example input
fn main() { println!("{:#x}", crc32(b"12345678")); }
In the above example, the correct output is
0x9ae0daaf
Exercise 6.5: CRC in Rust
Use a CRC checksum function written in Rust in a C program
Requirements
- A C compiler
Steps
-
Change Cargo.toml to
[package] name = "crc-in-rust" version = "0.1.0" edition = "2021" [lib] name = "crc_in_rust" crate-type = ["dylib"] # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html [dependencies]
-
Expose an extern rust function
#![allow(unused)] fn main() { #[no_mangle] pub extern "C" fn crc32(...) -> ... { ... crc32_rust(...) } }
-
Create a C header file
crc_in_rust.h
#include <inttypes.h> // uint32_t, uint8_t #include <stddef.h> // size_t uint32_t crc32(const uint8_t data[], size_t data_length);
-
Use the rust
crc32
function in C#include <inttypes.h> // uint32_t, uint8_t #include <stddef.h> // size_t #include <stdio.h> // printf #include "crc_in_rust.h" int main() { uint8_t data[] = { 0,1,2,3,4,5,6 }; size_t data_length = 7; uint32_t hash = crc32(data, data_length); printf("Hash: 0x%d\n", hash); return 0; }
-
compile and run
$ clang main.c target/debug/libcrc_in_rust.so -omain $ ./main Hash: -1386739207
Exercise 6.6: QOI Bindgen
In this exercise, we will use cargo bindgen
to generate the FFI bindings for a C library. Bindgen will look at a C header file, and generate Rust functions, types and constants based on the C definitions.
However, the generated code will likely be ugly and non-idiomatic. To wrap a C library properly, good API design and documentation is needed.
Background
The image crate provides functionality for encoding, decoding and editing images in Rust. It supports many image formats, like JPEG, PNG and GIF, but also QOI. QOI is a "Quite OK Image format", which aims for fast encoding and decoding of images, while providing a file size similar to PNGs. In this exercise, we test if the image crate produces the same results when decoding QOI images as the QOI reference C library.
The QOI C library is a header-only library, which means the function implementations are included within the header file instead of in a separate C file. We've added a separate C file which includes the header to make it easier to compile and include the library in our Rust program.
6.6 Generating bindings
Prerequisites:
- A C compiler is installed on the system
- Bindgen, which can be installed with
cargo install bindgen-cli
Steps:
-
Create the Rust bindings:
bindgen qoi.h -o src/bindings.rs
-
Use a
build.rs
script to compile and linkqoi.h
. Createbuild.rs
and insertfn main() { cc::Build::new().file("qoi.c").compile("qoi"); // outputs `qoi.a` }
And add this section to your
Cargo.toml
[build-dependencies] cc = "1"
-
Create
src/lib.rs
with the contentspub mod bindings;
. This will make thebindings
module available inmain.rs
. -
Run
cargo check
to verify everything is compiling correctly.
6.6 Inspecting our bindings
In the generated bindings.rs
file we find this signature for the qoi_read
C function from QOI:
#![allow(unused)] fn main() { extern "C" { pub fn qoi_read( filename: *const ::std::os::raw::c_char, desc: *mut qoi_desc, channels: ::std::os::raw::c_int, ) -> *mut ::std::os::raw::c_void; } }
Some observations:
- The definition is inside an
extern "C"
block, and has no body. Therefore, this function is marked as an extern, and Rust expects it to be linked in. - The function is marked
pub
, meaning we can import and use it in other modules (likemain.rs
in our case) - We can deduce the behavior somewhat from the type signature:
filename
is a C string with the name of the QOI file we want to readdesc
describes some metadata about the image, the function will write to thisqoi_desc
struct. This struct was also generated by bindgen:#![allow(unused)] fn main() { #[repr(C)] #[derive(Debug, Copy, Clone)] pub struct qoi_desc { pub width: ::std::os::raw::c_uint, pub height: ::std::os::raw::c_uint, pub channels: ::std::os::raw::c_uchar, pub colorspace: ::std::os::raw::c_uchar, } }
channels
is the number of channels the image has: either 3 for RGB images, or 4 for RGBA images (which also have an alpha channel for transparency). For this exercise, we will assume the images have an alpha channel.- The return value is a void pointer. If the function has successfully read the pixel data from a QOI image, then this pointer should point towards the pixel data.
- As the types are raw C types, it can be a hassle to call it directly from Rust.
We will deal with the last point by writing a nice Rust wrapper around the generated bindings.
6.6 Writing our wrapper
To make the qoi_read
function easier to use, we would like to write a wrapper that takes a path and returns an image buffer:
#![allow(unused)] fn main() { fn read_qoi_image(filename: &Path) -> ImageBuffer<Rgba<u8>, &[u8]> { todo!() } }
To implement this wrapper, there are a couple of challenges that need to be solved:
- We need to turn the path into a C string. Hint: we can use
std::ffi::CString::new
to create a C string from a sequence of bytes, and the most convenient way to turn the path into bytes is to first get theOsStr
from it. We can then pass the C string as a pointer. - We need to provide a
qoi_desc
, this struct can be imported from the bindings. Pass a mutable reference to an instance of this struct to the function. - After calling
qoi_read
, we need to turn the returned void pointer into an image buffer.- First, we should check if the returned void pointer
is_null()
. If it is null, something has gone wrong with reading the image. - Next, we need to determine the length of the returned pixel data. Assuming the image has an alpha channel, we have 4 bytes for every pixel in the image. The number of pixels in the image can be determined from the
qoi_desc
struct. - Now we can turn our void pointer into a
&[u8]
. We can cast our void pointeras *const u8
first. Next, we usestd::slice::from_raw_parts
with the previously calculated length. - Finally, we can use
ImageBuffer::from_raw
to construct our image buffer.
- First, we should check if the returned void pointer
To try out our wrapper, we can try to read a QOI image and export it as a PNG:
fn main() { let image = read_qoi_image(Path::new("image.qoi")); image.save("image.png").unwrap(); }
If implemented correctly, this should produce a nice picture!
Now that we can decode images using the QOI reference C library, we can test if the image crate produces the same results with the following unit test:
#![allow(unused)] fn main() { #[cfg(test)] mod tests { use crate::read_qoi_image; use std::path::Path; #[test] fn test_qoi_read() { let filename = "image.qoi"; let image = image::open(filename).unwrap().into_rgba8(); let my_image = read_qoi_image(Path::new(filename)); assert_eq!(image.width(), my_image.width()); assert_eq!(image.height(), my_image.height()); assert!(image.pixels().eq(my_image.pixels())); } } }
If you add this test to main.rs
and run it with cargo test
we should see:
running 1 test
test tests::test_qoi_read ... ok
6.6 Freeing the pixel data
When working with data from C, we are responsible for deallocating the memory once we are done using it. Some C libraries might provide a separate function to clean up data structures. For QOI, we instead have to call libc::free
to free the memory, as indicated by the documentation of the qoi_read
function:
The returned pixel data should be free()d after use.
To make sure someone using our wrapper does not forget to free the memory, we can implement the Drop
trait to automatically call libc::free
when the variable goes out of scope.
- First, create a wrapper
struct QoiSlice { ptr: NonNull<u8>, desc: qoi_desc }
, which holds the image buffer. - Next, implement the
Drop
trait forQoiSlice
to free the memory:#![allow(unused)] fn main() { impl Drop for QoiSlice { fn drop(&mut self) { todo!(); // call libc::free here using a pointer to the image buffer } } }
- To make this
QoiSlice
usable in anImageBuffer
, we have to implement theDeref
trait:#![allow(unused)] fn main() { impl Deref for QoiSlice { type Target = [u8]; fn deref(&self) -> &Self::Target { todo!() // create a slice from the ptr and lenght using `slice::from_raw_parts()` } } }
- Now update the
read_qoi_image
function to return an instance ofImageBuffer<Rgba<u8>, QoiSlice>
.
6.6 Uninitialized memory
There is one more trick: our current function initializes the qoi_desc
struct with zeros (or whatever values you put there while creating an instance of the struct). This is wasteful because the extern function will overwrite these values. Because the extern function is linked in, the compiler likely does not have enough information to optimize this.
For a relatively small struct such as qoi_desc
, this is not much of a problem. However, for larger structures or big arrays, this can make a serious impact on performance.
If we look at the LLVM IR, the intermediate representation which is generated and optimized before it gets turned into assembly code, we can see that it did not optimize away the initialization of the struct with values. Here we see it uses memset
to initialize the desc
with zeros before calling qoi_read
:
call void @llvm.memset.p0.i64(ptr noundef nonnull align 4 dereferenceable(10) %desc.i, i8 0, i64 10, i1 false), !noalias !142
%pointer.i = call noundef ptr @qoi_read(ptr noundef nonnull %t.0.i.i, ptr noundef nonnull %desc.i, i32 noundef 4) #17, !noalias !142
(The LLVM IR can be generated using cargo rustc --bin qoi-bindgen --release -- --emit=llvm-ir=llvm-ir.ll
, then search for @qoi_read
in llvm-ir.ll
)
The solution is to use std::mem::MaybeUninit
:
#![allow(unused)] fn main() { let mut desc = MaybeUninit::uninit(); let pointer = unsafe { qoi_read(filename.as_ptr(), desc.as_mut_ptr(), 4) }; let desc = unsafe { desc.assume_init() }; }
The MaybeUninit
type is an abstraction for uninitialized memory. The .uninit()
method gives a chunk of uninitialized memory big enough to store a value of the desired type (in our case qoi_desc
will be inferred).
6.6 Safety documentation
At the moment, the safety of your program relies on the context you have for the wrapped C library.
To ensure somebody modifying your library later does not break any of your assumptions you should always document what you assumed when writing the unsafe code
.
Add the following clippy lint to the top of your lib.rs
and main.rs
, run cargo clippy
and document your assumptions:
#![allow(unused)] #![deny(clippy::undocumented_unsafe_blocks)] fn main() { }
6.6 Bonus: Idiomatic interface
The current project is quite bare, you could improve that.
Structure
Currently, all your logic lives in main.rs
where it cannot be used as a library.
Move your types and functions over to lib.rs
and only expose the necessary functionality to the user.
Run cargo doc --open
to inspect what a user would see.
Error handling
Currently, we use panicking to handle errors.
This is problematic since it does not offer the user to handle those errors.
Change read_qoi_image()
to return a Result
instead.
And change possible error cases to return an Err()
instead of panicking.
Also consider which cases cannot possibly happen because of the guarantees you can make, and use expect()
to panic in that case.
Conclusion
In this exercise we saw how we can generate bindings to a C library with bindgen. The generated bindings are a bit difficult to work with, as they are unsafe and rely on C types. We've discussed how we can create nice wrappers around the generated bindings to deal with all these C types and to make them safer to work with.
Exercise 6.7: TweetNaCl Bindgen
Use cargo bindgen
to generate the FFI bindings. Bindgen will look at a C header file, and generate rust functions, types and constants based on the C definitions.
But the generated code is ugly and non-idiomatic. To wrap a C library properly, good API design and documentation is needed.
tweetnacl-bindgen
Making rust bindings for the tweetnacl C library
Exercise: implement crypto_hash_sha256_tweet
Below you find instructions for using bindgen and wrapping crypto_hash_sha512_tweet
. Follow the instructions, then repeat the steps for crypto_hash_sha256_tweet
Instructions
Prerequisites:
- a C compiler is installed on the system
- bindgen, install with
cargo install bindgen-cli
Steps
-
Create the rust bindings:
bindgen tweetnacl.h -o src/bindings.rs
-
Use
build.rs
to compile and linktweetnacl.c
. Createbuild.rs
and insertfn main() { cc::Build::new() .file("tweetnacl.c") .compile("tweetnacl"); // outputs `libtweetnacl.a` }
And add this section to your
Cargo.toml
[build-dependencies] cc = "1"
-
Create
src/lib.rs
with the contentspub mod bindings;
. This will make thebindings
module available inmain.rs
. -
Run
cargo check
to verify everything is compiling correctly. -
By default building will generate a bunch of warnings. we can turn those off by replacing our build.rs with
fn main() { cc::Build::new() .warnings(false) .extra_warnings(false) .file("tweetnacl.c") .compile("tweetnacl"); // outputs `libtweetnacl.a` }
and adding this line at the top of
src/bindings.rs
:#![allow(unused)] #![allow(non_upper_case_globals)] fn main() { }
Inspecting our bindings
In the generated bindings.rs
file we find this signature for the crypto_hash_sha512_tweet
C function from tweetNaCl:
#![allow(unused)] fn main() { extern "C" { pub fn crypto_hash_sha512_tweet( arg1: *mut ::std::os::raw::c_uchar, arg2: *const ::std::os::raw::c_uchar, arg3: ::std::os::raw::c_ulonglong, ) -> ::std::os::raw::c_int; } }
Some observations
- The definition is inside of an
extern "C"
block, and has no body. Therefore this function is marked as an extern, and rust expects it to be linked in. - The function is marked
pub
, meaning we can import and use it in other modules (likemain.rs
in our case) - We can deduce the behavior from the type signature:
arg1
is the output: a mutable pointer to a sequence of bytesarg2
is the input: a constant pointer to a sequence of bytesarg3
is a length (unclear of what)- the return value is probably an error code
- These are raw C types, which makes it a hassle to call directly from rust.
We will deal with the last point by writing some nice rust wrappers around the generated bindings.
In rust we bundle a pointer to a sequence of elements and its length in a slice. We could write the signature of our own rust wrapper function as:
#![allow(unused)] fn main() { pub fn crypto_hash_sha512_tweet(out: &mut [u8], data: &[u8]) -> i32 { todo!() } }
Modelling with types
But by looking at the tweetNaCl source code we can see that the contract is a bit stronger:
- the output is always 64 bytes wide (64 * 8 = 512)
- we only ever return
0
int crypto_hash(u8 *out,const u8 *m,u64 n)
{
u8 h[64],x[256];
u64 i,b = n;
FOR(i,64) h[i] = iv[i];
crypto_hashblocks(h,m,n);
m += n;
n &= 127;
m -= n;
FOR(i,256) x[i] = 0;
FOR(i,n) x[i] = m[i];
x[n] = 128;
n = 256-128*(n<112);
x[n-9] = b >> 61;
ts64(x+n-8,b<<3);
crypto_hashblocks(h,x,n);
FOR(i,64) out[i] = h[i];
return 0;
}
The rust type system can model these invariants: We can explicitly make the output 64 elements long by using a reference to an array. Furthermore we can drop the return type if there is nothing useful to return.
#![allow(unused)] fn main() { pub fn crypto_hash_sha512_tweet(out: &mut [u8; 64], data: &[u8]) { todo!() } }
But even better, we can return the output array directly:
#![allow(unused)] fn main() { pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] { todo!() } }
The compiler will turn this signature into the one we had before under the hood. Returning the value is more idiomatic and convenient in rust, and with modern compilers there is no performance penalty.
In detail: The C ABI mandates that any return value larger than those that fit in a register (typically 128 bits nowadays) are allocated on the caller's stack. The first argument to the function is the pointer to write the result into. LLVM, the backend used by the rust compiler has specific optimizations to make sure the function result is written directly into this pointer.
Writing our implementation
Allright, with the signature worked out, we can write the actual implementation.
We can reach the bindings from main.rs
with e.g.
#![allow(unused)] fn main() { tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet(a,b,c); }
Here tweetnacl_bindgen
is the name of the project, specified in the package
section of the Cargo.toml
[package]
name = "tweetnacl-bindgen"
Then bindings
is the module name (the file src/bindings.rs
is implicitly also a module) and finally crypto_hash_sha512_tweet
is the function name from the original C library.
On to the implmentation. Extern functions are considered unsafe in rust, so we will need an unsafe block to call ours.
#![allow(unused)] fn main() { pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] { unsafe { tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet( todo!(), todo!(), todo!(), ); } } }
Next we can pass our argument: we turn the slice into a pointer with .as_ptr()
, and get the length with len()
. The length needs to be cast to the right type. In this case we can use as _
where rust will infer the right type to cast to.
#![allow(unused)] fn main() { pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] { unsafe { tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet( todo!(), data.as_ptr(), data.len() as _, ); } } }
Next we create an array for the return value, pass a mutable pointer to this memory to our extern functin, and return the array.
#![allow(unused)] fn main() { pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] { let mut result = [ 0; 64 ]; unsafe { tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet( &mut result as *mut _, data.as_ptr(), data.len() as _, ); } result } }
And we're done: an idiomatic rust wrapper around the crypto_hash_sha512_tweet
!
Uninitialized memory
There is one more trick: our current function initializes and zeroes out the memory for result
. That is wasteful because the extern function will overwrite these zeroes. Because the extern function is linked in, the compiler likely does not have enough information to optimize the zeroing out away.
The solution is MaybeUninit
:
#![allow(unused)] fn main() { use std::mem::MaybeUninit; pub fn crypto_hash_sha512_tweet(data: &[u8]) -> [u8; 64] { let mut result : MaybeUninit<[u8; 64]> = MaybeUninit::uninit(); unsafe { tweetnacl_bindgen::bindings::crypto_hash_sha512_tweet( result.as_mut_ptr() as *mut _, data.as_ptr(), data.len() as _, ); result.assume_init() } } }
The std::mem::MaybeUninit
type is an abstraction for uninitialized memory. The .uninit()
method gives a chunk of uninitialized memory big enough to store a value of the desired type (in our case [u8; 64]
will be inferred).
We can look at the LLVM IR to verify that 1) the initialization with zeroes is not optimized away and 2) using MaybeUninit does not initialize the array.
Below is a call site of our crypto_hash_sha512_tweet
function that zeroes out the memory. Indeed, we see a memset
that sets all the bytes to 0. (also not that our wrapper function actually got inlined)
%result.i = alloca <64 x i8>, align 1
%0 = getelementptr inbounds <64 x i8>, <64 x i8>* %result.i, i64 0, i64 0
call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(64) %0, i8 0, i64 64, i1 false), !alias.scope !8, !noalias !11
%_2.i = call i32 @bindings::crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull "foobarbaz", i64 9)
In constrast, the version with MaybeUninit
just calls our extern function without touching the memory at all:
%result.i = alloca <64 x i8>, align 1
%0 = getelementptr inbounds <64 x i8>, <64 x i8>* %result.i, i64 0, i64 0
%_3.i = call i32 @bindings::crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull "foobarbaz", i64 9), !noalias !6
Full LLVM IR
define i8 @call_with_maybeuninit() unnamed_addr #1 personality i32 (i32, i32, i64, %"unwind::libunwind::_Unwind_Exception"*, %"unwind::libunwind::_Unwind_Context"*)* @rust_eh_personality {
start:
%result.i = alloca <64 x i8>, align 1
%0 = getelementptr inbounds <64 x i8>, <64 x i8>* %result.i, i64 0, i64 0
call void @llvm.lifetime.start.p0i8(i64 64, i8* nonnull %0), !noalias !2
%_3.i = call i32 @crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull getelementptr inbounds (<{ [9 x i8] }>, <{ [9 x i8] }>* @alloc1, i64 0, i32 0, i64 0), i64 9), !noalias !6
%1 = load <64 x i8>, <64 x i8>* %result.i, align 1, !noalias !7
call void @llvm.lifetime.end.p0i8(i64 64, i8* nonnull %0), !noalias !2
%2 = call i8 @llvm.vector.reduce.add.v64i8(<64 x i8> %1)
ret i8 %2
}
define i8 @call_without_maybeuninit() unnamed_addr #1 personality i32 (i32, i32, i64, %"unwind::libunwind::_Unwind_Exception"*, %"unwind::libunwind::_Unwind_Context"*)* @rust_eh_personality {
start:
%_4 = alloca <64 x i8>, align 1
%0 = getelementptr inbounds <64 x i8>, <64 x i8>* %_4, i64 0, i64 0
call void @llvm.lifetime.start.p0i8(i64 64, i8* nonnull %0)
call void @llvm.memset.p0i8.i64(i8* noundef nonnull align 1 dereferenceable(64) %0, i8 0, i64 64, i1 false), !alias.scope !8, !noalias !11
%_2.i = call i32 @crypto_hash_sha512_tweet(i8* nonnull %0, i8* nonnull getelementptr inbounds (<{ [9 x i8] }>, <{ [9 x i8] }>* @alloc1, i64 0, i32 0, i64 0), i64 9)
%1 = load <64 x i8>, <64 x i8>* %_4, align 1
%2 = call i8 @llvm.vector.reduce.add.v64i8(<64 x i8> %1)
call void @llvm.lifetime.end.p0i8(i64 64, i8* nonnull %0)
ret i8 %2
}
Unit 7.1 - Scientific Computing with Rust
Exercise 7.1.1: PyO3
Write a custom python extension using PyO3.
Python is a convenient and popular language, but it is not fast. By writing complex logic in faster languages, you can get the best of both worlds. PyO3 makes it extremely easy to write and distribute python extensions written in Rust.
PyO3 and SIMD
PyO3 makes it easy to write python extensions in rust. The code for this exercise is a skeleton, taken from the PyO3 documentation.
you should be able to run this example like so from the repository root:
folkertdev@folkertdev ~/t/teach-rs (mod-g)> cargo build -p pyo3-simd
Compiling pyo3-simd v0.1.0 (/home/folkertdev/tg/teach-rs/exercises/G/4-pyo3)
Finished dev [unoptimized + debuginfo] target(s) in 0.19s
folkertdev@folkertdev ~/t/teach-rs (mod-g)> cp target/debug/libpointwise_simd.so pointwise_simd.so
folkertdev@folkertdev ~/t/teach-rs (mod-g)> python3
Python 3.8.5 (default, May 27 2021, 13:30:53)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pointwise_simd
>>> dir(pointwise_simd)
['__all__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'sum_as_string']
>>> pointwise_simd.sum_as_string(4,5)
'9'
>>>
Our goal is to implement pointwise addition of two python lists of floats in rust using SIMD instructions.
- hook up the
pointwise_sum
pyfunction, it should callpointwise_sum_simd
. It is easiest to useVec<f64>
in the interface. - next run
cargo test -p pyo3-simd
. This should compile, but the test fails. Use the given simd functions and pointer offsets to implement thepointwise_sum_simd
correctly. - verify that this works from python.
If that succeeded: congrats, you can now write arbitrary python extensions, and speed up python code. Rust and PyO3 make this really straightforward.
Teacher's companion to teach-rs
If you have decided to try teach-rs for your students, you will probably run into two problems:
-
As an academic, you may feel your own practical knowledge of Rust is lacking.
-
You will have to make a selection of subjects to fit practical constraints.
So what parts of teach-rs should you invest time in to teach to your students? And how much time is required?
We assume you have a clear idea of your learning outcomes, and your target audience. Teach-rs can be used for first-year students at university, for master's students, or even for an internal training for senior engineers at your software company, but obviously different groups would require a different approach!
Teach-rs is a modular course
We have defined particular tracks, which consists of selections of modules that go well together given a certain learning outcome and target audience, for example teach-rs focussed on Web programming or teach-rs focussed on Embedded Devices; you can see the full list of tracks here:
Finer-grained modularity
If you want finer-grained control over content selection, we have structured every module into a few topics. A topics is defined by
a set of slides and recommended exercises. You can construct your own modules by selectiong topics. We have defined dependencies between
topics; for example, if you pick the basic-syntax
topic you should also select the why-rust
topic. These dependencies ensure that
you should still end up with a coherent course.
If you take this route, however, you have to take more responsibility that the study load remains balanced, as (unlike with modules),
topics don't have a fixed study time associated with them. For example (again), the why-rust
topic will require less time (and has no
partical exercises attached to it) than the basic-syntax
topic. Since teach-rs is in active development, we cannot give
time estimates per topic and are focussing more on balancing the study load for the full course and the pre-defined tracks.
Overview of modules and topics
General modules of the Rust course can be divided into "common" and "specialized" modules. The common ones will be useful for every track (for example, "Language Basics") whereas others can be viewed as optional (for example, "Rust for Web").
Module 0 (introduction) is recommended in full for every course, since it outlines the motivation for learning Rust, and broadly introduces its features. Module A contains all topics related to language features.
Reference material
Several online resources exists that can provide valuable background material for you (or your students).
- The Rust Programming Language, online book, also available as paperback
- Rust by Example, online reference with idiomatic examples of Rust code
- Rust for Rustaceans, book that assumes some prior knowledge and dives into more advanced topics
- Learning Rust with Entirely Too Many Linked Lists, focussing on Rust's ownership and borrowing rules
- Rust Design Patterns, a collection of idiomatic Rust patterns (and anti-patterns)
- The Rust Reference, a reference suitable for finding explanations for finer points of syntax and semantics.
- The Rustonomicon, a reference that is specific to low-level aspects, application binary interfaces and
unsafe
code.
Exercise solutions
Teach-rs is provided without answers to exercises. If you have need of those, please contact us.
Contributing
Need help? Have questions? Say hi in our Matrix channel:
Great that you want to contribute to teach-rs! Here's some ways in which you can help improve the project.
Improving the material
If you see anything you think can easily be improved with a small pull request, such as fixing typos, fixing links or re-formulating sentences, please open a PR directly.
If you want to propose a structural change to one or more modules, please open an issue first, so we can discuss your ideas. Please clearly state your proposed changes and your motivation for suggesting the changes. In case an improvement proposal is accepted, a PR with the changes referring to the relevant issue can be made.
Adding a module
Teach-rs is intended to be used as a basis for custom courses that either are about Rust itself or use Rust to teach some other concept. To that end, the teach-rs project aims to incorporate modules on diverse topics.
If you feel teach-rs should cover a topic that is currently not covered, you can propose adding a module by opening an issue. State the main goal and the learning objectives of the module, as well as the covered topics and proposed exercises. In case an addition proposal is accepted, a PR with the proposed changes referring to the relevant issue can be opened.
Have a look at the structure of the current slides and exercises to get a better understanding of how modules are organized.
Trying it out on your students
Teach-rs is distributed under a Creative Commons license, so even if you don't have the time to contribute to this repository directly, you are free to teach from it and to adapt its material to suit your students' needs. You can also help the development of teach-rs by communicating your experiences---what modules did your students like, which ones did they struggle with?
Just open an issue and don't forget to mention (without going into personal details) what the background of your students is and which parts of the course you used.