Sebastian KaiserA personal engineering notebook

The notebook Learning Rust

Processing JSON with Rust

As I was learning Rust, I wanted a practical project to build some syntax muscle memory. Loading JSON seemed like a useful place to start, so I chose a nested dataset from the Tokyo Olympics and tried to turn it into a medal table.

We’ll follow the steps from reading the nested fields to counting medals for each country and ranking the results. A country lookup that needs a fallback gives us one detail to work through along the way.

The exercise uses serde_json to read athletics data from the Olympic Games Tokyo 2020. It ranks countries by gold medals, then silver, then bronze, and prints the first five.

TextPrint the top five countries and the medals they won.Link
United States of America MedalCount { g: 7, s: 12, b: 7 }
Italy MedalCount { g: 5, s: 0, b: 0 }
Kenya MedalCount { g: 4, s: 4, b: 2 }
Poland MedalCount { g: 4, s: 2, b: 3 }
Jamaica MedalCount { g: 4, s: 1, b: 4 }

I chose an excerpt of the athletics data set because it has more to work through than a small, uniform example. The data is deeply nested, and the countryObject field needs closer inspection.

Expanded JSON tree showing pageProps, gameDiscipline, events, awards, and a participant with countryObject named Kenya.
Figure 1: Structure of our JSON data

The expanded path runs through pageProps, gameDiscipline, events, awards, and participant to countryObject.

To produce the medal table, we need to read medalType and the country information from each award, as shown above.

Reading the data Two shapes, one country lookup. Participant objects from the worked example below. These are illustrative data, not excerpts from the Olympic dataset.

Object field

{
  "countryObject": {
    "name": "Japan"
  }
}

Read countryObject.name when countryObject is an object.

Fallback field

{
  "country": {
    "name": "Italy"
  }
}

Otherwise read country.name. Here, countryObject is absent.

A detail in the data

countryObject is not always an object. The lookup tests is_object() before choosing the field and otherwise falls back to country.

The code

Serde_json allows us to map JSON data to enums and this is what we are going to do. Since we also want to count the different types of medals we define a struct called MedalCount.

rustLink
#[derive(Debug, PartialEq, Eq, Deserialize)]
enum Class {
    #[serde(rename = "GOLD")]
    Gold,
    #[serde(rename = "SILVER")]
    Silver,
    #[serde(rename = "BRONZE")]
    Bronze,
}

#[derive(Default, Debug, PartialEq, Eq, PartialOrd, Ord)]
struct MedalCount {
    g: usize,
    s: usize,
    b: usize,
}

Use a medal type to select its counter

Serde maps medal values to Class. For counting, I want each variant to select the corresponding counter in MedalCount, so the accumulation loop can stay short.

Rust’s Index trait supplies read access to a selected counter; IndexMut supplies mutable access. The implementations below map each Class variant to the corresponding g, s or b field.

rustLink
impl Index<Class> for MedalCount {
    type Output = usize;

    fn index(&self, class: Class) -> &Self::Output {
        match class {
            Class::Gold => &self.g,
            Class::Silver => &self.s,
            Class::Bronze => &self.b,
        }
    }
}

impl IndexMut<Class> for MedalCount {
    fn index_mut(&mut self, class: Class) -> &mut Self::Output {
        match class {
            Class::Gold => &mut self.g,
            Class::Silver => &mut self.s,
            Class::Bronze => &mut self.b,
        }
    }
}

As far as I know, serde_json does not support mapping of nested data to struct via some form of dot notation. Luckily, if data is not mapped to a predefined struct than the JSON data can still be deserialized into a Value, which then allows us to get the nested data we want.

Reading the codeFrom nested results to a country ranking
  1. Walk the events and their awards

    pageProps.gameDiscipline.events[]each event’sawards[]

  2. Extract one medal and one country

    MedalmedalType

    Deserialize to Class: Gold, Silver, or Bronze.

    Countryparticipant

    Use countryObject.name when countryObject is an object; otherwise use country.name.

    Country lookup in the listing

    One (Class, country) pair per award.

  3. Fold the pairs into a counter for each country

    HashMap<String, MedalCount> keeps the running totals. IndexMut selects the field to increment.

    g golds silverb bronze
    Counting in the listing
  4. Rank the countries, then take five

    Descending by gold, then silver, then bronze. The calling code uses take(5) to select the result—not a sum of all medals.

    Comparison in the listing

Follow the three operations in this listing:

rustExtract country names, count medals, and rank the results.Link
fn get_class_country_tuple<'a>(values: &'a Vec<Value>) -> Vec<(Class, String)> {
    let get_country = |participant: &'a serde_json::Value| -> &'a serde_json::Value {
        let mut country_key = "countryObject";
        if !participant[country_key].is_object() {
            country_key = "country";
        }

        &participant[country_key]["name"]
Note

Extract the country name. The lookup uses countryObject when it is an object, otherwise country.

Context
    };

    let to_tuple = |v: &'a Value| {
        let class = serde_json::from_value::<Class>(v["medalType"].to_owned()).unwrap();
        let country = get_country(&v["participant"]).as_str().unwrap().to_string();
        (class, country)
    };

    values.iter().map(to_tuple).collect()
}

// returns a projection on the medals per country data
pub fn get_medals_per_country(&self) -> Result<Projection, Box<dyn std::error::Error>> {
    let to_medal_country_tuple =
        |v: &serde_json::Value| get_class_country_tuple(v["awards"].as_array().unwrap());

    let group_by_country = |mut acc: HashMap<_, _>, x| {
        let (class, country): (Class, String) = x;

        // remember the IndexMut trait, we used it here for indexing
        // If there is no entry for a country yet, we create one and
        // initialize the medal count to g: 0, s: 0, b: 0.
        // The fields will be incremented by one based on the indexing
        // operations defined earlier.
        // So, the value "Gold" will be deserialized to the enum variant
        // Class::Gold which will increment the field `g` of teh struct
        // MedalCount.
        acc.entry(country).or_insert(MedalCount::default())[class] += 1;
Note

Accumulate the medals. Each country gets a counter; IndexMut selects the gold, silver, or bronze field to increment.

        acc
    };

    let events = self.0.json["pageProps"]["gameDiscipline"]["events"]
        .as_array()
        .unwrap();

    let mut medals_per_country = events
        .into_iter()
        .flat_map(to_medal_country_tuple)
        .fold(HashMap::new(), group_by_country)
        .into_iter()
        .collect::<Vec<(_, _)>>();

    medals_per_country.sort_by(|a, b| b.1.cmp(&a.1));
Note

Rank the results. The comparison orders by gold first, then silver, then bronze—not by the total medal count.


    Ok(Projection(medals_per_country))
}
rustLink
#[derive(Eq, PartialEq)]
struct Projection(Vec<(String, MedalCount)>);

impl Projection {
    fn empty() -> Self {
        Self(vec![])
    }

    fn take(self, n: usize) -> Self {
        Self(self.0.into_iter().take(n).collect())
    }

    fn get(&self) -> &Vec<(String, MedalCount)> {
        &self.0
    }
}

Having all the components in place, we can finally implement the logic we wanted:

rustLink
fn load_data() -> Result<(), Box<dyn std::error::Error>> {
    let url =
        "https://raw.githubusercontent.com/sekaiser/rust-sample-parse-json/main/athletics.json";
    let db = AthleticsDb::from_url(Url::from_str(url)?)?;

    let mut last_top5 = Projection::empty();
    loop {
        let medals_per_country = db.get_medals_per_country()?;
        let top5 = medals_per_country.take(5);

        if top5 != last_top5 {
            for e in top5.get() {
                println!("{} {:?}", e.0, e.1);
            }

            last_top5 = top5;
        }
        thread::sleep(time::Duration::from_secs(2));
    }
}

Follow the medal pipeline

One worked example, from five awards to a ranked medal table. This small, invented dataset is not the Olympic results; the values below illustrate one pass through the article’s extraction, counting, and ranking steps.

Example input · 5 awards

TextLink
{
  "pageProps": {"gameDiscipline": {"events": [
    {"awards": [
      {"medalType":"GOLD","participant":{"countryObject":{"name":"Japan"}}},
      {"medalType":"SILVER","participant":{"country":{"name":"Italy"}}},
      {"medalType":"BRONZE","participant":{"country":{"name":"Italy"}}},
      {"medalType":"GOLD","participant":{"countryObject":{"name":"Japan"}}},
      {"medalType":"GOLD","participant":{"country":{"name":"Kenya"}}}
    ]}
  ]}}
}
1. Extract country and medal

Flatten the events into five awards. Read countryObject.name when the country is an object there; otherwise use country.name.

TextLink
Japan → gold
Italy → silver
Italy → bronze
Japan → gold
Kenya → gold
2. Count medals per country

Group the five awards into three countries, incrementing the matching medal counter for each award. Japan’s two gold awards become one row.

TextLink
Japan: gold 2, silver 0, bronze 0
Italy: gold 0, silver 1, bronze 1
Kenya: gold 1, silver 0, bronze 0
3. Rank and take the top five

Compare gold first, then silver, then bronze. Kenya’s one gold places it ahead of Italy’s silver and bronze. There are only three countries, so taking the top five retains all three; no tie-breaker is needed.

TextLink
1. Japan — 2 gold
2. Kenya — 1 gold
3. Italy — 1 silver, 1 bronze

What I learned

This exercise gave me a practical way to work with serde_json and Rust’s traits. The country lookup, medal counters and ordering rule each have a place in the code, so I can inspect one step without working through the whole pipeline again.

If the iterator chain is the part you want to understand better, The Iterator Is the State Machine follows how adapters keep state and borrow data. It works through a separate telemetry example.

Next steps

For the publishing side of this article, Org-mode publishing showcase collects the examples I use to check code listings, stored results and other exported features.