TIP: Avoiding a Subtle Trap in Rust's For-loop Move Syntactic Sugaring

, 202 words, 1 minutes read

When processing data as a read operation from any persistence layer of choice, bulk of the time is spent as:

It is tempting to write this, but note that for-loops in Rust are just syntactic sugar for IntoIterator. This means the contents are moved into the block

let collection1 = vec![1, 2, 3];
let collection2 = vec![collection1];

for inner in collection2 {
    // contents are _moved_ into the iterator here.
    for items in inner {}
}

Being consumed means that we cannot access them at a later point. This does not work:

// this is the naive way of accessing the inner iterator, which also causes a move due to
// Rust's for-loop being syntactic sugar of the IntoIterator trait.
for inner in collection2 {
    // contents are _moved_ into the iterator here.
    for items in inner {}
}
// collection2.iter();
// ^
// |__ we are now unable to iterate over it, due to the previous move.

Consider this approach, which I have extended to a HashMap.

for r in collection2.iter().flat_map(|r| r.iter()) {
    println!("value: {}", r);
}
// notice that we did not move the contents of the collection above.
let _ = collection2.iter().collect::<Vec<&Vec<i32>>>();

// the same applies for any other iterator; here's the same example with a HashMap:
let inner = HashMap::from([("".to_owned(), 8)]);
let collection = HashMap::<String, HashMap<String, i32>>::from([("a".to_string(), inner)]);

for (k, v) in collection.iter().flat_map(|(_, r)| r.iter()) {
    println!("value: {}", v);
}

I have applied this to my latest project which processes accounts within a ledger system. These are queried from the persistence layer into a Vec<AccountQueryRecord> but we have double nesting to group them by

let mut accounts_hashmap: HashMap<
        Currency,
        HashMap<AccountType, (Vec<AccountQueryRecord>, AccountsTotalAsDecimal)>,
    > = HashMap::new();

The same approach above lets us partition the processed results and iterate over the original records fetched from the DB without needlessly re-allocating them on the heap.

Did you find this useful? Feel free to email me and share your thoughts!

If you made it this far, have a great day!

#rust