March 2026 Updates
March 30, 2026
March has come to an end, and it has been a very busy March. I have
three updates to share. First WiNDC.GMS is a new repository designed to
replace the old windc_build repository. Second,
GDXInterface.jl is a new package that allows you to load
GDX files directly in Julia without the need to install GAMS. Finally,
there are some updates to MPSGE.jl that you may find useful.
WiNDC.GMS
I have created a new repository called WiNDC.GMS. The goal of this repository is to make it easier to use and share WiNDC data and code. This can be considered a replacement for the old windc_build repository, which was very large (~1 GB), difficult to pinpoint how the data was modified, and could be difficult for new users to start using. WiNDC.GMS works to fix each of these issues.
Let’s walk through the process of using the core module
of windc_build. First, you must download the
core data from the WiNDC website and place it into the
correct directory. Then, you run the build script in the
core module. This runs through several steps:
- Load the supply and use tables and split them into national-level parameters.
- Calibrate the national-level parameters. This step uses the
proprietary solver
CPLEXwhich has caused issues for some users. - Make adjustments to negative parameters values.
- Verify the benchmark calibration of the national model.
- Disaggregate the national-level parameters to the regional level. This is done over several steps and files.
- Verify the benchmark calibration of the regional model.
- Export a compiled GDX file containing the regional parameters.
While this process doesn’t take long, it takes time. It is also
fairly opaque, it’s difficult to know how the data is being modified at
each step. WiNDC.GMS simplifies this process. First, there
are three explicit modules: national,
regional, and household. Each module is
structured identically, with a build.gms script that runs
through the following steps:
- Download the compiled data from the WiNDC website. This allows you to specify a specific version of the data, which should help with reproducibility. This GDX file contains a minimal set of parameters.
- Create GDX files containing the parameters for the module. This is
done using the
load/create*.gmsscripts. In particular, thecreate_bea_data.gmsscript creates all aggregated parameters (likegross_output). We have three variants of the data:bea,windc, andlegacy_windc, we’ll discuss the differences below. - Verify the benchmark calibration of the module using an MPSGE model.
The data used in this process is assembled and exported from Julia using the three WiNDC Julia packages: WiNDCNational.jl, WiNDCRegional.jl, and WiNDCHousehold.jl. In the next section we discuss how the data was transferred from Julia to GDX.
Each module has three variants of the data: bea,
windc, and legacy_windc. The bea
variant is designed to exactly match the Julia parameters and sets. It
uses NAICS codes for sectors/commodities and long names for parameters,
i.e.xa0Intermediate_Demand instead of id0. Both
the windc and legacy_windc variants use WiNDC
labels as sectors/commodities and have parameter names that match the
old windc_build repository. However, the windc
variant has parameter domains that match the new WiNDC structure, while
the legacy_windc variant is designed to match
windc_bulid.
Each module and data variant have scripts to load the data and run
updated MPSGE models. For example, if you are using a project that uses
legacy_windc, you only need to copy the files
legacy_windc_load_data.gms and
legacy_windc_model.gms into your project. Each file has
options to point to a specific data file, which are fully documented
within each file.
GDXInterface.jl
The number one question I get from users transitioning from GAMS to
Julia is “how do we transfer data from GAMS to Julia?”. My old answer to
this was to use the gams.transfer package in Python to open
a GDX and save the sets/parameters as CSV files. I created a small repository
with example code to do this.
This process works, but it is cumbersome. It is a two step process that is fairly error prone. If your GDX file doesn’t have the correct sets/parameters, you won’t know until you try to load the CSV files in Julia. It also requires you to have Python installed and be familiar with using it.
It would be much better if we could load the GDX file directly in
Julia. This is now possible with the new GDXInterface.jl
package. This package provides a simple interface to load/unload GDX
files directly in Julia without the need to install GAMS. I’ve created a
small repository with the code I used to export the data for
WiNDC.GMS using GDXInterface.jl. You can find
it here,
note that there is no README file and I haven’t tested this code on
other machines. This is intended as a proof-of-concept for using
GDXInterface.jl to export data from Julia to GDX.
This package is still in development, but it is already very useful.
Once this gets released I plan to updated the WiNDC data
repositories so that users can export their data directly from Julia to
GDX.
MPSGE Updates
In the February update, I discussed some updates to MPSGE.jl. In
particular, the unit_cost_functions. This is now available
on the main branch of MPSGE.jl. You can test it by adding the main
branch of MPSGE.jl to your project:
pkg> add MPSGE.jl#mainWe are planning to release a new version of MPSGE.jl soon. There is
one more pull request we’d like to merge before the release. This final
pull request reworks the back-end of the generate_report
function. The new version should be much faster and more robust. It will
also allow you to generate reports for specific variables, for
example:
generate_report(M[:PX])will generate a report for the just the PX variable.
April
In April, I plan to switch back to the data side of things. The WiNDCHousehold repository is still in development. It currently only works for 2024, I’d like to get it working from at least 2017 onward (and ideally get 2011 and 2014 operational as well). I may also touch on Canadian data and general bilateral trade data.