Skip to contents

External inputs

Source: R/whep_inputs.R
whep_inputs.Rd

The information needed for accessing external datasets used as inputs in our modeling.

Usage

whep_inputs

Format

A tibble where each row corresponds to one external input dataset. It contains the following columns:

  • alias: An internal name used to refer to this dataset, which is the expected name when trying to get the dataset with whep_read_file().

  • board_url: The public static URL where the data is found, following the concept of a board from the pins package, which is what we use for storing these input datasets.

  • version: The specific version of the dataset, as defined by the pins package. The version is a string similar to "20250714T123343Z-114b5". This version is the one used by default if no version is specified when calling whep_read_file(). If you want to use a different one, you can find the available versions of a file by using whep_list_file_versions().

Source

Created by the package authors.

Frozen predecessor-pipeline references

Four aliases in whep_inputs are not outputs of this package: primary_prod, commodity_balance_sheet, processing_coefs and feed_intake. They are the 2025-07-14 snapshot of the predecessor R-script pipeline this package replaced, kept frozen so inst/scripts/compare_global_whep.R can benchmark whep against it. No package function reads them, their schema is the old one (CamelCase, name-keyed, no polity columns) and they stop in 2021, so reading one warns.

Their numbers are not what current code produces, so they are references, not substitutes. primary_prod (2,443,516 rows, 1961–2021) against a fresh build_primary_production() over the same 2015–2021 window:

  • The pin has no slaughtered_heads row in any of its years; the build emits 12,195 over 2015–2021. Three item_prod codes are pin-only (378, 773, 1163; 1,332 rows) and three appear in no pin year at all (1051, and the modelled grassland items 3001 and 3002).

  • 317,587 rows against the build's 328,242 for the window, a 3.2% shortfall.

  • Its 2020–2021 fodder harvested area is carried forward from 2019 — 85.93 Mha in each year, 468 country-item series over 96 areas, equal to 2019 to the last digit — where the build emits no fodder row at all after 2019, because eu-agridb-fodder stops in 2019, faostat-production-old in 2013, and faostat-production carries none of the 16 fodder items.

  • Even in a year both cover they disagree: 2019 fodder harvested area is 85.93 Mha in the pin against 90.42 Mha from the build.

Build the current series with the matching build_*() or get_*() function instead.

The batch pin on the build path

Two further aliases were published in that same 2025-07-14 batch — crop_residues and bilateral_trade, alongside the four above, between 12:33:43Z and 12:33:50Z. Unlike the four, these two are read on the default build path, which is why no warning is attached to them: every commodity balance build reads them, so a warning at the read is noise rather than information. bilateral_trade has since been regenerated by this package (#1122), so crop_residues is the only one still registered at its 2025-07-14 version.

They did not share a provenance despite sharing a timestamp. Each was established separately (#1054), because the timestamp alone establishes nothing.

crop_residues is predecessor output

Read by get_primary_residues(), and from there by build_commodity_balances(). All 475,688 of its Product rows equal the primary_prod pin's tonnes values exactly — no key unmatched on either side, no value differing at a relative tolerance of 1e-6 — so it is a downstream artifact of the same predecessor run, carrying that run's production series into the commodity balance.

Its residue quantities are those production numbers times a residue-to-product ratio that varies by year, and times a recovery rate: the pin holds the residue the predecessor recovered from the field, not the residue produced (#1195). The predecessor's code (Global/R/crop_npp.r, then afsetools::residue_use()) writes each row as

product * kg_residue_kg_product_FM * HI_changes(region, year) * residue_dm_product_dm(region, category) / residue_dm_product_dm("West Europe", category) * recovery_rates(region, category),

with region and category keyed as in residue_recovery.csv. Rebuilt from the predecessor's own coefficient tables, that formula reproduces all 425,934 pin rows whose region it can key, at a relative tolerance of 1e-6. The year variation is HI_changes alone: a harvest-index change factor per region, given at eight anchor years from 1910 to 2000, linear in between and flat after 2000 (East Europe 1.10 in 1962 to 1.00 in 2000), which the predecessor attributes to Krausmann et al. (2013), PNAS 110:10324, Table M2 (assumed, unverified). It is a regional historical adjustment, not a yield response, and after 2000 the ratio of every region and category is constant. That is why biomass_coefs$kg_residue_kg_product_FM alone reproduces only 3,189 of the 472,790 comparable keys. Of the 33,411 keys that carry a residue of exactly 0, 30,620 are cells whose recovery rate is 0 (fodder crops in seven of the eight regions; roots and tubers, cassava, sugar beet and dry beans in West Europe and in North America and Oceania) and 1,919 are crops whose biomass_coefs ratio is 0.

WHEP ships the recovery rate and the regional ratio the predecessor used (both equal in all 160 cells), but not the HI_changes table. So get_primary_residues() undoes the recovery exactly, dividing by the same rate, but cannot recompute the predecessor's residue from production. It then re-keys the regional ratio on Wirsenius's own region membership, and estimates cereal residue (#1448), and that of the non-cereal crops with their own IPCC Table 11.2 line (#1518), from the pin's production and harvested area instead of reading it.

It is not a small input. The pin holds 7.63 Gt of recovered residue for 2010 (Straw 3.60 Gt, Other crop residues 2.49 Gt, Firewood 1.54 Gt) and 327.7 Gt over 1961–2021, which get_primary_residues() turns into 7.64 Gt (Straw 3.92 Gt, Other crop residues 2.82 Gt, Firewood 0.90 Gt) and 366 Gt of residue produced. 3,998 of its 249,095 output rows carry NA polity columns because the pin is name-keyed.

Driving the same residue model off a fresh get_primary_production() would move those numbers. Measured for 2010 at (area_code, item_prod): 1.958 Gt of current production sits on 6,425 keys the pin never sees (809 Mt of it primary crops, including the modelled temporary-grassland item), 64.8 Mt on 296 keys is pin-only, and of the 8,010 shared keys 538 disagree, putting 1.102 Gt — 11.01% of the pin's shared-key mass — more than 1% apart. The largest class is rice, 673.9 Mt in the pin against 463.3 Mt fresh, a ratio of 0.6876: current code puts rice on a milled-equivalent basis while the pin's is paddy, which is a basis difference rather than an error, since straw scales with the field crop. The second is fodder, where the current build is close to twice the pin on every forage and silage item, the pin predating that work. Replacing the pin therefore means choosing a residue model, which is a science decision and not a refresh. For cereals that model is chosen by get_primary_residues(cereal_residue = ), and for the non-cereals with an IPCC line by non_cereal_residue =; both still run on the pin's production.

bilateral_trade is built by build_detailed_trade()

Read by get_bilateral_trade() and by the live-animal branch of build_commodity_balances(). The registered version, 20261001T135651Z-ca7b3, is build_detailed_trade()'s output verbatim, with its defaults and extend_time = FALSE, run on the faostat-trade-bilateral pin 20260407T095142Z-b3f81 (#1122): 10,038,220 rows over 1986–2021, polity-keyed, carrying 64,450,133,319 tonnes and 87,840,869,640 head.

The 2025-07-14 version it replaces, 20250714T123347Z-2c392, held the same FAOSTAT Detailed Trade Matrix values in the predecessor pipeline's name-keyed schema — against the raw pin over 2010, all 296,642 shared tonnes keys and all 5,102 shared Head keys agree exactly — but without FAOSTAT's 1000 Head rows: 76,141,882 thousand head of live broiler chicken, turkey, duck, rabbit and goose trade over 1986–2021, which PR #1113 recovered in the producer. The regenerated pin carries 7.50x its 11,707,083,640 head and 22.5% less than its 83,147,095,412 tonnes, all of that tonnage difference in CBS items 5001-5003, which no commodity balance carries: item 1293's unbacked tonnage, screened by method_unbacked_quantity (#1023), and three CBS names with no item_cbs_code. On the tonnes that do reach a trade matrix the two versions differ by 0.056%. .clean_bilateral_trade() reads either schema.