Skip to contents

Build primary production dataset

Source: R/build_production.R
build_primary_production.Rd

Construct the full primary production dataset from raw FAOSTAT inputs. This is a convenience wrapper that chains the pipeline steps:

  1. .read_production() — read & reformat FAOSTAT data.

  2. .fix_production() — apply Global-ported corrections.

  3. .dedup_production() — keep one value per key across sources.

  4. .qc_production() — flag data-quality anomalies on the surviving (deduplicated) values.

Usage

build_primary_production(
  start_year = 1850,
  end_year = 2023,
  smooth_carry_forward = FALSE,
  example = FALSE,
  show_duplicates = FALSE,
  historical_data = NULL,
  federation_land = c("none", "successor_union"),
  land_method = c("present_day", "historical_polity"),
  fodder_split = c("fao_mix", "equal"),
  .raw_data = NULL
)

Arguments

start_year

Integer. First year to include. Default 1850.

end_year

Integer. Last year to include. Default 2023.

smooth_carry_forward

Logical. If TRUE, carry-forward tails are replaced with a linear trend. Default FALSE.

example

Logical. If TRUE, return a small hardcoded example tibble instead of reading remote data. Default FALSE.

show_duplicates

Logical. If TRUE, return only the rows that have competing sources in wide format (one column per source) for diagnostic comparison. Default FALSE.

historical_data

Optional harmonized historical production rows to add before the LUH2 historical extension. May be a data frame or a path to a parquet/csv file. Required semantic columns are year, item_prod_code, unit, value, and one of area_code or polity_area_code. Names such as item_prod_name, item_cbs_name, and source are used when present; WHEP item and area tables fill canonical names where possible. Observed historical rows are retained, and LUH2 proxy filling can use them as anchors. Rice supplied here is assumed to be on a paddy (rough-rice) basis and is multiplied by the paddy-to-milled extraction rate, because WHEP's rice item is milled equivalent throughout; pre-divide by that rate if the series is already milled. Default NULL.

federation_land

Character. How the pre-1962 LUH2 back-cast reaches an area whose territory is a dissolved federation. LUH2 land use is keyed on present-day ISO3, so 15 Belgium-Luxembourg, 51 Czechoslovakia, 228 USSR and 248 Yugoslav SFR have no land record of their own.

  • "none" (default, current published behaviour) leaves them unmatched; their pre-1962 production is not back-cast at all and the build warns.

  • "successor_union" rebuilds each federation's land series as the sum of its successor states' LUH2 land, resolved from the successor relation published in polities. This back-casts 14.3% more of the 1961-62 production tonnage and therefore moves published pre-1962 values. Under land_method = "historical_polity" it reaches only the federations the historical table has no polygon for, and their rows keep the LUH2_* source label, not LUH2_polity_*.

land_method

Character. Which borders the pre-1962 ha half of tonnes = ha * t_ha is measured on. The yield half is historical either way and is untouched by this argument.

  • "present_day" (default, current published behaviour) reads the luh2-areas pin, which is LUH2 land pre-aggregated to present-day ISO3, so a row labelled with the 1961 entity is measured on the borders that entity has today.

  • "historical_polity" measures it with build_historical_land_areas(): gridded LUH2 summed inside the polygon of the polity area_code resolved to in that year. It moves published pre-1962 values, needs sf and terra, and reads gridded LUH2 for every back-cast year, so it is minutes of extra work.

fodder_split

Character. How an EU AgriDB fodder area is divided among the FAOSTAT items that share its Eurostat label (seven items share "Other plants harvested green from arable land", four "Other root crops n.e.c."). Either way the items together get exactly the reported area; until whep#654 each item could get all of it.

  • "fao_mix" (default) splits it in proportion to FAOSTAT's own item areas in that year, or in the nearest years that report them. A label FAOSTAT never gives an item area for in that country is split evenly.

  • "equal" gives every item sharing the label the same share.

.raw_data

Optional tibble with the same structure as the output of the internal .read_production() step. When supplied, the remote-data read is skipped entirely and the pipeline starts from .fix_production(). Columns required: year, area, area_code, item_prod, item_prod_code, item_cbs, item_cbs_code, live_anim, live_anim_code, unit, value, source. fao_flag is used when present and completed as NA when it is not. Default NULL.

Value

A tibble with the columns of get_primary_production() plus fao_flag: year, legacy numeric area_code, numeric polity_area_code, reporting_polity_code, reporting_polity_name, reporting_polity_has_geometry, item_prod_code, item_cbs_code, live_anim_code, unit, value, source, and fao_flag. Item names can be recovered via add_item_prod_name() and related helpers. When show_duplicates = TRUE, returns a wide tibble with one column per source showing the competing values.

fao_flag is FAOSTAT's own observation-status code for the value ("A" official, "E" estimated, "I" imputed, "M", "X"), and it describes the number in value rather than the row's item or area. It is NA wherever the number is not one FAOSTAT published under a flag, which is most rows that are not unit == "tonnes" or unit == "ha":

  • WHEP's own yields (t_ha, t_LU, t_head) are ratios it computes, so FAOSTAT's separate Yield flag is not a statement about them;

  • LU and heads are livestock-unit conversions summed over an animal's products, and slaughtered_heads is one FAOSTAT count split across CBS items by shares;

  • a gap-filled, back-cast, imputed or reconstructed value (fill_linear, imputed_yield, LUH2_*, EuropeAgriDB, DM_yield_estimate, Estimated) is WHEP's estimate, not a reported figure – and a FAOSTAT_prod row can still be one of these, because source is resolved per key while the flag is resolved per quantity;

  • a value summed or averaged from parts whose flags disagree is dropped rather than credited to one of them (whep#581).

Measured on a real 2010-2013 build: 45.5% of rows carry a flag, 91.7% of tonnes rows and 88.2% of ha rows.

Green fodder area is reconstructed, and source says how. FAOSTAT reports fodder tonnage only (faostat-production-old, to 2012); EU AgriDB reports area and yield (to 2019), and its rows read "EuropeAgriDB". A fodder area held flat past its series' last observation reads "DM_yield_estimate_carried_forward", one held flat before its first "DM_yield_estimate_carried_backward", and one interpolated between two observations "DM_yield_estimate". From 2013 every non-EU fodder area is its 2012 value carried forward: measured on the real inputs, 75.0 of the 100.0 Mha of green fodder area in 2013, against 2.2 of 99.5 Mha in 2012. A fodder series therefore changes what it is made of at 2012/2013 with no change in level (whep#1027). That is why check_series_jumps() finds nothing at 2013 in fodder area per country (no area of 100 is flagged), and why it finds the break at once in the yearly share of fodder area whose source is a carried one (0.022 to 0.750, a ratio of 33.5).

Examples

build_primary_production(example = TRUE)
#> # A tibble: 10 × 13
#>     year area_code polity_area_code reporting_polity_code reporting_polity_name 
#>    <dbl>     <dbl>            <int> <chr>                 <chr>                 
#>  1  1912       165              165 PAK-WP-1949-1971      West Pakistan (1949-1…
#>  2  2012       112              112 JOR-1946-2025         Jordan                
#>  3  1943        41               41 CHN-1950-2025         China (PRC)           
#>  4  1979        45               45 COM-1975-2025         Comoros               
#>  5  1910       141              141 MNG-1921-2025         Mongolia              
#>  6  1867        90               90 GIN-1958-2025         Guinea                
#>  7  1939        15               15 BLX-1850-1999         Belgium-Luxembourg    
#>  8  1935       211              211 CHE-1800-2025         Switzerland           
#>  9  1937         9                9 ARG-1902-2025         Argentina             
#> 10  2000         9                9 ARG-1902-2025         Argentina             
#> # ℹ 8 more variables: reporting_polity_has_geometry <lgl>,
#> #   item_prod_code <chr>, item_cbs_code <dbl>, live_anim_code <chr>,
#> #   unit <chr>, value <dbl>, source <chr>, fao_flag <chr>