Rust Iterators: A Guide

By the time you walk out of here, you should understand what iterators are good for, how they work internally, and how to create your own!

Iterators are a fairly central concept to Rust. If you're looping over something, you're very likely already using an iterator. If you're transforming collections, you probably should be using them. If your function returns a lazily evaluated sequence of things, you should consider returning an iterator - especially if that sequence could be lazily evaluated.

WYWL: What you will learn#

We'll take a look at how iterators are implemented, how to iterate over a collection, what sorts of iterators exist in the standard library, usage and common patterns for transforming data, and finally a few examples of useful crates that provide their functionality via powerful iterators.

Prerequisites#

Skill-wise, you'll ideally have an understanding of

  • structs and enums,

  • the Option<T> generic type,

  • traits, and

  • just a pinch of closures,

I highly recommend having some sort of environment to run snippets of code in. The simplest thing to use is the Rust Playground. If you'd like a local environment, refer to The Book for guidance on setting that up.

What is an iterator?#

Essentially, an iterator is a thing that allows you to traverse some sort of a sequence. Note that since Rust's iterators are lazy, this sequence could be generated on the fly - you could just as well traverse an existing array of finite length or create an iterator that keeps spewing out random numbers infinitely.

Laziness in programming is this general idea of delaying a computation until it's actually needed. A lazy iterator doesn't need to know all the elements it's going to return when it's first initialized - it can compute every next element when/if it's asked for.

The Iterator trait#

In Rust, iterators are typically implemented using the Iterator trait. All we need to implement that trait for our custom type is provide an associated type Item (this is the type of the elements of the sequence, returned by the iterator) and the next method.

Let's try and implement an iterator over numbers from 1 to 10.

The next method is expected to yield the next element of the sequence. It takes a mutable reference to self in case we need to keep track of some state between next calls, which is normally the case. We'll soon find it useful.

The return type of next is Option<Self::Item>. If an iterator is finite, it needs to return None to indicate it has no more elements to return. Right now, this iterator will immediately finish, not yielding any items. Let's fix this.

This should be pretty self-explanatory. Every call to next increments self.currentand yields it until it grows beyond 10.

Looping#

So we have an iterator. Let's use it. The most basic thing we can do is simply loop over all of its elements:

There's actually an Iterator implementation for the Range type in Rust! A Range is what you get when you type something like 1..5. Instead of writing the above custom iterator, we could have simply done this:

IntoIterator, iter() and iter_mut()#

Sometimes you'll want to provide the user of your code the ability to iterate over your type, but without that type itself being an iterator. This will be the case with collections. If you implement your own vector type, you probably don't want that type to needlessly hold an extra iteration variable just in case someone wants to iterate over it.

What we want is a way to create an iterator out of the collection. There are three mechanisms you'll typically see.

  • The IntoIterator trait is implemented for the type and provides you with the into_iter method. This one consumes the data and wraps it in an owning iterator.

  • The iter method is defined directly on the type. This method will borrow the data immutably and return an iterator that provides immutable references.

  • The iter_mut method is defined directly on the type. This method will borrow the data mutably and return an iterator that provides mutable references.

If we want to iterate over a vector of chars, we could do something like this:

If we leave out the into_iter() call, Rust will call that method implicitly anyway.

This implicit call is important to keep in mind. Since into_iter() consumes the data, we cannot use the original vector later.

One solution would be to explicitly call v.iter() so that the iterator is borrowing instead. Another is to provide a reference to v rather than the owned value.

This way, the compiler can no longer implicitly call into_iter() since it doesn't get an owned value. It gets an immutable reference, so the best it can do is implicitly call iter() on it - and that's what we want.

Then there's mutability. Following the same pattern, here Rust will implicitly call iter_mut() and give us an iterator that is mutably borrowing.

Adapters#

If all you could do with iterators was loop over them with the for keyword, they wouldn't be all that useful. But there is a plethora of adapters that transform an iterator into another kind of iterator, altering its behavior.

Most adapters you'll work with are in the standard library and exist as methods provided for the implementers of the Iterator trait. There are some crates out there (such as itertools) that provide extra adapters via extensions.

We're going to go through a few useful adapters, but I highly recommend taking a look at the full list in Rust documentation.

We can construct an infinite iterator using std::iter::repeat. It would be a bad idea to iterate over it directly.

If we wanted to print only a few 1s, however, we can use the take adapter for that.

What take does under a hood is wrap the Repeat iterator in a Take wrapper. Take is also an iterator, but this one finishes after returning a set number of elements.

A very typical feature of functional programming is the ability to map, that is to apply a function to every element of a sequence.

If we wanted to find only elements that fulfill a certain criterion, we can use the filter adapter.

Collecting#

We know how to turn a collection into an iterator. How do we turn an iterator into a collection? Enter the collect method, provided by the Iterator trait.

We could try to convert from a vector to an iterator and back.

This, however, produces an error.

What Rust is telling us here is that it doesn't know what we're trying to collect into. collect has a generic return type and could give you a number of things: a vector, a linked list, a string, etc. You could even create a custom type that can be collected into.

To let Rust know what concrete type we want collect to return, we can use the turbofish syntax.

We can make this slightly shorter. The compiler should be able to figure out that we want a vector of chars, specifically, and not a vector of integers. When filling out type parameters, we can use an underscore to tell Rust, "Figure this part out yourself!"

In a case like this, you might find it tidier to add a type annotation to the variable declaration instead. The whole thing will then look like this:

Using collect, we can convert an array of chars into a string.

We could also collect an iterator of tuples (where the first element needs to be hashable) into a HashMap:

Things that can be collected into implement the FromIterator trait. That means this behavior is extendable! Check out the trait's docs to see which types can be collected into and how to implement new ones.

Common patterns#

We now have some idea of how iterators work and some operations we can perform on them. Let's put it together and see some typical use cases.

Transform and collect#

Let's say we store customer data in Customer structs, which include a customer's name, e-mail address, and how much they owe us.

Then let's say we have a list of such customers. We're tasked with producing a vector of all debtor e-mails so we can send them a generic reminder.

How do we do it? The nice, idiomatic way is to get an iterator over customers, apply some adapters that will filter and transform the data, and then collect that back into a vector.

Finding things in collections#

Given the same Customer struct as above, and the same vector of customers, we can search the customer data for a specific person. There's no useful method defined directly on the Vec<T> type, but there is a find method defined on the Iterator type.

What if we'd like to get the position of an element in the vector? Things get a little trickier, but create an iterator.

We then have to enumerate it to keep track of positions. enumerate will wrap every element in a tuple of formthe the (position, element).

Then we have to change our find closure a little to account for the items now being tuples.

Finally, once we unwrap the Option<(usize, Customer)>, we still have to extract the position component of the tuple, which is the 0th one.

Splitting strings#

The str type comes with a split method that yields an iterator over chunks of that string. All you have to provide is a pattern to split by - commonly a char, a &str or a String.

For example, you could get all the words of a phrase this way:

And then you could transform them and collect them into a new String:

The rev method is an adapter that reverses an iterator!

Examples of third-party iterators#

These are some examples of third-party libraries providing some functionality via iterators.

  • walkdir - walkdir allows us to traverse a directory easily - it yields an iterator over a chosen directory's entries (files and subdirectories)

  • logos - lexers created with logos return iterators over tokens

  • csv - when reading a CSV file, we get an iterator over its records

Conclusion#

This just about exhausts the core concepts and basic usage. Hopefully, you should now be able to not only effectively transform collections, but (with some practice) also identify where providing iterators would make sense in your code.

Next steps#

One thing to do now would be to simply read the module-level documentation for std::iter, and take a look at the provided methods of the Iterator trait. There are some useful tools to discover there that we didn't cover here!