Patchwork - MapConverter

Level 7
Joined
Jun 30, 2017
Messages
50

Introduction​


Patchwork - MapConverter is a Node.js console application used to convert between warcraft III's map binaries and their textual representation. This allows for easier way of distributing map's source code through git and potentially being able to resolve merge conflicts for stuff that is typically edited through the World Editor, like terrain or Object data.

Additionally, patchwork can extract trigger files into multiple files replicating the category tree from trigger editor into your filesystem and also compile this same directory tree into a valid triggers/custom scripts files.

Requirements​


To install and use this tool, you will require the following:
  • Node.js v21 or higher
  • npm package manager installed (bundled with Node.js installation)
  • node added to system PATH variable (also an option in Node.js installation)
After node.js installation just simply run this command in your terminal: npm i -g patchwork-mapconverter

Warning

In case you're updating from 0.0.7 version you must convert your GUI triggers back into war3map format before updating!!!

How to use​


Running npx patchwork-mapconverter will display the options and commands this tool has to offer:
1691333044952.png


To use this tool, you must provide it a triggerdata.txt file, which you can obtain by opening wc3 archive in CascView. It can be found on _retail_/UI path.
To provide this file you've extracted to the tool, you must active the following option npx patchwork-mapconverter -td <filePathToTriggerData.txt>

Note: This tool works for maps saved in latest World Editor, I can't guarantee they will work for map files created/saved using older World Editor versions.

I believe most of these descriptions offer good enough explanation of what specific options do. However, I will point out a few particularly useful features:
Smart imports option works differently depending on command:

war2json​

When this command is chosen along with this option, smart imports will copy all files that are defined inside war3map.imp (world editor's imports registry file) and paste them into the <output>/<importsFolderName> path, effectively separating the import files from the other map files.

json2war​

When this command is chosen along with this option, smart imports will copy all the files from <input>/<importsFolderName> into <output> but retaining the relative path that was inside the <importsFolderName>, furthermore the option also generates its own war3map.imp file into the output path, populating it with all those copied files within <importsFolderName>, eliminating the need of doing imports by using the imports manager in World Editor.
Compose triggers options also works differently depending on command:

war2json​

In this mode, the tool will take war3map.wtg and war3map.wct files (GUI triggers, and custom scripts) and export them into <output>/<sourceFolder> while retaining the relative path that was found in the trigger editor. Trigger Library and Category elements are converted into folders, while everything else turns into files. The kind of file depends on the configured extension. (Note: changing the file extension for those sub-options will not cause the tool to change the format these triggers are exported as, they will always be JSON)

Furthermore, for every category the tool will generate a .ini file which specifies a few category options and most importantly, the order of variables/custom scripts/triggers inside said category, because one cannot rely on alphabetical ordering of files (which is done for folders) when it comes to custom scripts due to map loading process.

json2war​

In this mode, the tool will take all files and folders found inside <input>/<sourceFolder> and convert them into categories, triggers, variables, scripts, etc.. retaining the relative paths and trigger editor element ordering stored in .ini files, if they're provided. If the ordering is not provided, the order of these elements is at the discretion of whatever the Node.js runtime decides (safe to assume that it will be random)
With this option, you can specify a file with Regex rules of which files should the tool ignore, this file works for both war2json and json2war, if you want the ignore file to work for only 1 of the commands, consider giving the tool a filepath that doesn't exist, or having 2 separate files for each command. (Note: this doesn't ignore tool-exported files, such as the imports file or triggers file, in case of smart imports or compose triggers options enabled.)
Using this option you will have to supply a triggerdata.txt file which can be acquired from CascView opening the game, or a modified triggerdata.txt like GrapesOfWath's Enhanced GUI triggers. The tool and the World editor uses this file in order to read the triggers file.
Possible options are .json, .yaml, .toml, more are coming later.
Note: This tool cannot process map in file mode, maps must be saved in folder mode in order for this tool to process the map's files.

A nice suggestion​

In case of using VSCode, my suggestion is to have a .vscode/launch.json with following content:
JSON:
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "map2json",
            "program": "patchwork-mapconverter",
            "request": "launch",
            "runtimeExecutable": "npx",
            "skipFiles": [
                "<node_internals>/**"
            ],
            "type": "node",
            "args": [
                "-d",
                "-ct",
                "-si",
                "-td",
                "./.vscode/triggerdata.txt",
                "war2json",
                "./build.w3m",
                "./map"
            ]
        },
        {
            "type": "node",
            "request": "launch",
            "runtimeExecutable": "npx",
            "name": "json2map",
            "skipFiles": [
                "<node_internals>/**"
            ],
            "program": "patchwork-mapconverter",
            "args": [
                "-d",
                "-ct",
                "-si",
                "-td",
                "./.vscode/triggerdata.txt",
                "json2war",
                "./map",
                "./build.w3m"
            ]
        }
    ]
}

Doing so, allows you to have a quick way of executing this tool's commands whenever you have something that needs to be tested, or simply for making a map for general public.
1691340732584.png

map2json task should executed after every World Editor change, like modifying the terrain or defining new units in object editor, so that their JSON counterparts can get these updates.
json2map task should be executed after every code or import change, before testing a map or saving the map to a file for public release.

json2map task, with the forementioned configuration in tasks.json file will generate a build.w3m folder, which can be opened in World editor to test/edit the map.
1691340936158.png


Another advice is to have this folder added to your .gitignore file, since the idea of this tool is to have only textual representation of these files in your repository.
Furthermore, in patchwork.ignore file, you could add these files:
Code:
war3map.wtg
war3map.wct
war3map.j
war3map.lua

So that when executing map2json task, it doesn't touch your source code. And there's no need to have the compiled .j/.lua files in your repo.

If you have any issues you can comment on this thread, or if you find any bugs you can raise an issue on GitHub.

Happy coding/map making! :)
  • All illegal foldername characters that can be typed as trigger elements in trigger editor are now being replaced by another character:
TypeScript:
function safeReplaceTriggerName(name: string): string {
  return name
    .replaceAll('/', '-')
    .replaceAll('\\', '-')
    .replaceAll(':', ';')
    .replaceAll('*', '+')
    .replaceAll('?', '!')
    .replaceAll('"', '\'')
    .replaceAll('<', '(')
    .replaceAll('>', ')')
    .replaceAll('|', '_')
}
  • Doodad flags have been fixed (fixed my mistake regarding bitwise operators)
  • Triggers can now be converted from/to YAML and TOML options (-ge option)
  • Doodad flags have been fixed (begone outdated definitions!) (fixes doodad's Z height problem)
  • Empty patchwork.ignore file no longer causes problems.
  • Fixed trigger composer and import composers not checking with file blacklist
  • Added support to convert map data to/from YAML and TOML (-mde option)
  • Added prettify option that works only for JSON format (-p option)
  • OE data which contains a \u0000 char (Curse ability's 'Crs' field) are now properly transferred between binary and json format
  • Scripted triggers will no longer sometimes be enabled when they're supposed to be disabled after converting JSON to war3map format
  • runOnMapInit now properly processed for scripted triggers
  • can now handle converted triggers
  • added previously unsupported things from .doo (and .doo.units) and .w3i:
    • random drop tables
    • game data set
    • game data version
    • use TFT
    • water tinting
    • terrain fog
    • map properties opened at least once
    • map was reduced from original terrain size
    • upgrades & tech blacklist
    • player all/enemy low/high priority bitmaps
    • waygate data
    • special doodads (doodads embedded into terrain)
  • imports will no longer have auto-prepended war3mapimported in the path
  • triggerdata.txt is now only required when converting GUI triggers
  • trigger comments which are using illegal slash and backslash will have those converted into '-' symbols
  • duplicate trigger comments will get numbers appended to their names
  • .wts will no longer lose non-ascii chars (using utf-8)
  • fixed converting disabled custom scripts from war3map file to FileSystem + those same custom scripts are now copied back into war3map file
  • GUI triggers JSON format has been changed in order to fix specific cases caused by WEU converter
Anyone updating from older versions must first convert their GUI triggers from JSON back into war3map format before proceeding with updating this program!!!
 
Last edited:
Level 26
Joined
Aug 18, 2009
Messages
4,099
I have converted a .w3a to json to see what the format looks like. This operation should not require a TriggerData.txt. The output file looks like

JSON:
{
    "original": {},
    "custom": {
        "A000:Amls": [
            {
                "id": "amcs",
                "type": "int",
                "level": 1,
                "column": 0,
                "value": 25
            }
        ]
    }
}

Does "column" stand for the data pointer (DataA, DataB, ...)? I am not sure about using <newId>:<baseId> as the key. You would have to parse these ids again if you wanted to know them and semicolons might also be used in an id. The advantage here is that this structure enforces that there can be no double entries, yet "A000:AHre" would also be a unique key, even though then there are two entries with the same new id. Imho it would make more sense and be more stable to write it like:

JSON:
{
    "original": {},
    "custom": {
        "A000": {
            "baseId": "Amls",
            "modifications: [
                {
                    "id": "amcs",
                    "type": "int",
                    "level": 1,
                    "column": 0,
                    "value": 25
                }
            ]
        }
    }
}

Original objects can use the id of the object they are modifying without specifiying a base id. And version 3 of the Warcraft format can have different sets of modifications per object: war3map(skin).w3* Modifications

The other reason why I am writing though is to note that to use the json effectively, it would be good to have standardized schemas. There is JSON Schema. This would allow for broad support of tools to consume and write the instances. There are also tools that can work with the schemas like parser generators.

The schema for object modification files could look similar to this:

JSON:
{
  "$schema" : "https://json-schema.org/draft/2020-12/schema",
  "$defs" : {
    "ObjectsChunk" : {
      "type" : "object",
      "properties" : {
        "objects" : {
          "type" : "array",
          "items" : {
            "type" : "object",
            "properties" : {
              "baseId" : {
                "type" : "string"
              },
              "mods" : {
                "type" : "array",
                "items" : {
                  "type" : "object",
                  "properties" : {
                    "dataPointer" : {
                      "type" : "integer"
                    },
                    "endToken" : {
                      "type" : "string"
                    },
                    "id" : {
                      "type" : "string"
                    },
                    "level" : {
                      "type" : "integer"
                    },
                    "value" : {
                      "anyOf" : [ {
                        "type" : "number"
                      }, {
                        "type" : "string"
                      } ]
                    },
                    "valueType" : {
                      "type" : "string",
                      "enum" : [ "INT", "REAL", "UNREAL", "STRING" ]
                    }
                  }
                }
              },
              "newId" : {
                "type" : "string"
              },
              "unknown" : {
                "type" : "array",
                "items" : {
                  "type" : "integer"
                }
              }
            }
          }
        }
      }
    }
  },
  "type" : "object",
  "properties" : {
    "customObjectsChunk" : {
      "$ref" : "#/$defs/ObjectsChunk"
    },
    "extended" : {
      "type" : "integer"
    },
    "format" : {
      "type" : "string",
      "enum" : [ "OBJ_0x1", "OBJ_0x2", "OBJ_0x3" ]
    },
    "originalObjectsChunk" : {
      "$ref" : "#/$defs/ObjectsChunk"
    }
  }
}
 
Level 7
Joined
Jun 30, 2017
Messages
50
I have converted a .w3a to json to see what the format looks like. This operation should not require a TriggerData.txt.
I am planning on finishing a rewrite of this tool, so the next version won't have this issue. Another reason why I'm doing a rewrite is because the original project that I cloned from does not translate all information; like unit droptables, waygates, a lot of .w3i information, and some other things I find commented in the code.

About the object id in JSON, I am perfectly fine with changing it to fit the JSON schema you provided. And if you have schemas for other files, I'd be happy to implement those designs as well.
 
Level 26
Joined
Aug 18, 2009
Messages
4,099
I am planning on finishing a rewrite of this tool, so the next version won't have this issue. Another reason why I'm doing a rewrite is because the original project that I cloned from does not translate all information; like unit droptables, waygates, a lot of .w3i information, and some other things I find commented in the code.
Yes, saw it on GitHub shortly after.

About the object id in JSON, I am perfectly fine with changing it to fit the JSON schema you provided. And if you have schemas for other files, I'd be happy to implement those designs as well.
That was only an example, it's not finished yet. It was generated from other code I am currently working on in exploration efforts and I will be discussing the schemas with Luashine and others before putting it in a separate repository. A question is also how faithful and permissive it should be. For instance, in the above case, the value type could be derived from the value and/or meta slk, so it's not necessarily necessary to include in the json, unless you want to be able to express any native object modification file. Maybe it could be optional. And there are still some fields we do not know or if all bits in a bit set are used.
 
Level 7
Joined
Jun 30, 2017
Messages
50
Update 0.1.0 changes:
  • can now handle converted triggers
  • added previously unsupported things from .doo (and .doo.units) and .w3i:
    • random drop tables
    • game data set
    • game data version
    • use TFT
    • water tinting
    • terrain fog
    • map properties opened at least once
    • map was reduced from original terrain size
    • upgrades & tech blacklist
    • player all/enemy low/high priority bitmaps
    • waygate data
    • special doodads (doodads embedded into terrain)
  • imports will no longer have auto-prepended war3mapimported in the path
  • triggerdata.txt is now only required when converting GUI triggers
  • trigger comments which are using illegal slash and backslash will have those converted into '-' symbols
  • duplicate trigger comments will get numbers appended to their names
  • .wts will no longer lose non-ascii chars (using utf-8)
  • fixed converting disabled custom scripts from war3map file to FileSystem + those same custom scripts are now copied back into war3map file
  • GUI triggers JSON format has been changed in order to fix specific cases caused by WEU converter
Update 0.1.1 changes:
  • runOnMapInit now properly processed for scripted triggers
Update 0.1.2 changes:
  • Scripted triggers will no longer sometimes be enabled when they're supposed to be disabled after converting JSON to war3map format

Anyone updating from older versions must first convert their GUI triggers from JSON back into war3map format before proceeding with updating this program!!!

To update, simply write: npm -g i patchwork-mapconverter in your command line
 
Last edited:
Level 10
Joined
Jul 18, 2005
Messages
323
Now that I have started using Patchwork a bit more, I have noticed that map2json and json2map doesn't retain information about Doodad's Z heights (from using Ctrl + Page Up or Crtl + Page Down). I don't have a MWE of this yet (it could just be the map I'm developing), but it would be nice to have that feature in a future build.
 
Level 10
Joined
Jul 18, 2005
Messages
323
Just a bump to attach my minimum working example of doodad Z-heights not being converted correctly when using Patchwork.
  • mwe-og.w3m is the map file I saved, with some doodads with modified Z-heights.
  • mwe is the output generated by Patchwork when I run npx patchwork-converter -d -ct -cse .j -si -td <path-to-trigger-data> war3json ./mwe-og.w3m/ ./mwe
  • mwe-conv.w3m is the map file created by Patchwork when I run npx patchwork-converter -d -ct -cse .j -si -td <path-to-trigger-data> json2war ./mwe/ ./mwe-conv.w3m
 

Attachments

  • mwe-root.7z
    15.5 KB · Views: 7
Level 7
Joined
Jun 30, 2017
Messages
50
Hello, Merry Christmas and a Happy New Year :)

So, we've figured out what the cause of this is, and it turns out that our documentation of doodad flags wasn't correct.

(HiveWE wiki, already updated by Bogdan)
image_2025-01-03_133116491.png
The value these doodads had was 6 (which wasn't known at the time) which the program didn't know what it was, therefore it defaulted to 0.The 6 was actually 4 + 2, where the 4 was FixedZ option in the world editor:
image_2025-01-03_131042800.png


There was some further testing that basically discredited the previous flag definitions (e.g. visible, solid, etc...), and it turns out those were complete bogus. So the doo.json will have a change to reflect this.

JSON:
//before
      "flags": {
        "visible": false,
        "solid": false
      }

//after
      "flags": {
        "inUnplayableArea": false,
        "notUsedInScript": false,
        "fixedZ": false
      }

Old values will no longer be used, nor can they be directly migrated since the whole parsing was wrong, so it's best to map2json them from the binary files again.

Besides the doodad fix, prettify option now works for GUI triggers, and GUI triggers can also be converted to toml and yaml formats as well.
Empty patchwork.ignore file should no longer cause the program to crash.
 
Level 15
Joined
Aug 16, 2019
Messages
199
Your tool is truly outstanding, and I appreciate the effort that has gone into it!
I have a question regarding the parsed w3u data—I noticed that the unit table doesn’t contain a reference to the wts string for unit names. Is this a limitation of the parsing process, or is there a different way the names are linked to unit codes? I haven’t yet figured out how the names are associated—could you clarify how this connection works?

I see the unit id, I see that there is a comment in the wts file that goes to the name, but it doesn't seem like a connection

image.png
 
Last edited:
Level 7
Joined
Jun 30, 2017
Messages
50
Hello, after a quick glance, it seems that that particular data was moved over to war3mapSkin.w3u file

1738877372494.png
1738877395719.png


Edit: both the skin object data and the original object data use the exact same format, so there's no difference to patchwork, it's just that Blizzard decided to separate out some data when Reforged hit, presumably to be able to have different graphic modes supported in the same map?

It doesn't really explain why name of all things was moved there, but we won't really know the reason :D
 
Last edited:
Level 15
Joined
Aug 16, 2019
Messages
199
Thanks for your response! I didn't expect to see a reference to a name in the skin file and was consciously ignoring them... That’s really not obvious.

A small question—does the bidirectional parsing of files (war2json → json2war) with restarting the editor and re-saving not break the map?

Also, can wts strings be reused for insertion into w3* files?

And one more thing—does parsing allow viewing non-standard data based on the donor unit? To get the actual unit data in Reforged, you need to parse the SLK tables from the game itself, if I'm not mistaken?
 
Level 7
Joined
Jun 30, 2017
Messages
50
A small question—does the bidirectional parsing of files (war2json → json2war) with restarting the editor and re-saving not break the map?
It shouldn't, if you find such causes, you can report them to me and I'll take a look at it.

Also, can wts strings be reused for insertion into w3* files?
I'm like 90% sure it can, it's just that World Editor is lazy and generates a new string for every string entry. Which kinda does make sense, because languages can have the same word for something but have 2 different meanings, depending on context, and as such when translated would yield into 2 different words.
E.g:
- Karta (Croatian) -> Map (English) or Playing Card (English)

And one more thing—does parsing allow viewing non-standard data based on the donor unit? To get the actual unit data in Reforged, you need to parse the SLK tables from the game itself, if I'm not mistaken?
Patchwork only reads the binary data stored on the map, therefore, any unmodified data has to be fetched manually from the game's SLKs.
The only exception to this is converting GUI triggers, it will require you to provide it a triggerdata.txt file.
 
Level 15
Joined
Aug 16, 2019
Messages
199
Actually, your tool has enormous potential for use, even up to developing a full-fledged map editor.

I don't have such ambitions, but my goal is to use the parser for the automatic maintenance of my wiki project. However, even a minimal object editor could be useful—it would make grouping and balancing much easier.
 
Top