Do old tools still work for Reforged?

[Xardian]

Greetings, you wonderful hive community!

So, a little insight on my situation. I have been modding, modeling, texturing and loving wc3 for well over a decade now. I have been working on a project for about 9 years (with several do overs, because I was an unfocused teen and developing my skills)
And now since the beginning of last year I had been doing quite a bit of progress on my map project.

The situation though is this. I learned to model in mdlvis, and have no knowledge about, or access to official modeling tools.
So of course I was very hesitant to even get reforged, with all the negative press to it, and all the questionable decisions by Blizzard. And of course cause I am a notoriously broke individual and so the pricetag is a bit much for me.

But as my project is done completely with custom models, and custome textures, made by me and only a handful of assets from other creators, the option to have a more detailed render engine would be amazing.

But of course having made all of this with wc3s restrictions in mind, I am now at a point, where my models would be amazing compared to standart wc3 graphics, and work with the filesize limit. But where compared to reforged models, they would ofc fall flat, due to the lack of normal maps, and the amateurish, low-tech approach to texturing and modeling in general.
(With some unavoidable seams on the model, due to the lack of any on-model texture blending options)



So, as said, I do not have access to 3dsmax, no knowledge in blender (not to mention that there is no exporter for it afaik) and thus would remain restricted to the classic tools made for wc3.
I saw, the new models are still in mdx format, even though I think they added stuff like weightpainting for the rig, normal maps, gloss?...
I am a bit envious of the anti-aliasing and nice resolution on the reforged version too...

So is it worth it to get reforged at all? Do the old tools still work? What about the community? I read, that due to the release, there is quite a divide within the community, between people unable or unwilling to play reforged at all and people now unwilling to play maps that are below the reforged quality standart.

I will definitely continue working on my project, but question remains, if I should upgrade it to reforged or not.


Cheers!



PS: also an insight on upgrading models to reforged if possible would be grand! Like, do the new normal maps entirely replace the old lighting model? (Cause that would ofc mean normal glitches in mdlvis wouldnt be as big a deal, as I could just fix it through normal maps.) How easy is it to create normal maps from handdrawn textures for low poly models? Is it even possible? I remember from my time with Minecraft shaders, that there were options for that in that system.
(EDIT: After reading up some more, I saw, that there are a total of three new texture maps for the models. What comes to mind there though, is it done automatically by filename extension, or do the new models need to have their materials edited to include the new textures? As that ofc would making material work in Margos and Mdlvis impossible.)
How "off" do models without weight painting look in the new renderer? Are there ways to rig models for that, or people with access to 3dsmax able/willing to rig character models for other ppl?
(EDIT: Is it possible to create, rig and animate models in the classic tools and have them be "upgraded" through the official tools? And again, are there people able/willing to do that?)

I use gimp, krita, paint.net for textures and mdlvis+margos for modelling.

 

[deepstrasz]

@Retera please?

 

[Retera]

I have been modding, modeling, texturing and loving wc3 for well over a decade now.

Good, don't stop.

I learned to model in mdlvis, and have no knowledge about, or access to official modeling tools.

There are not official tools for Reforged yet to my knowledge. If it helps, once they are released they are most likely to be Maya plugins -- due to knowledge datamined from within the Reforged Beta client ("Maya Export Logs").

I saw, the new models are still in mdx format, even though I think they added stuff like weightpainting for the rig, normal maps, gloss?...

The new format uses the same extension. However, the new models will crash Magos and Mdlvis. You might be able to get a programmer to upgrade Mdlvis because the source code is public, but the documentation is all in Russian language(s). I do not know Russian languages.

So is it worth it to get reforged at all?

This is a question of personal judgement. I can't help you there. I preordered Reforged 50 times in the summer of 2019.

Do the old tools still work?

No. Reforged models, even the SD ones, have the .mdx extension but (at least the last time I checked) were still encoding in the "new MDX" format. So if you take the "Classic" footman model from Reforged, it would still crash Magos or Mdlvis most likely. (The HD "Reforged" footman, of course, will also not work in those tools).
So, because of that, I took my own tool and updated it.
The version from this thread is Reforged compatible, at least enough to open and save models as well as to open old models and then save them "into the Reforged format":
Retera Model Studio Reforged Hack

My tool has an animation editor and some other useful features but some users report that it is lacking functionalities from Mdlvis that they wish they had. I don't doubt it. Maybe later on I will have time to add more features. I don't really consider it to be finished, but I published it because I figured that *what I had* was already at a point where it would be useful for people.

also an insight on upgrading models to reforged if possible would be grand!

There are two menu options in my tool that relate to this issue. First off, when we talked about Reforged models and the old models, the difference inside the model file is a version number. The old models are version 800, and the new models are version 1000. This is made more confusing because of what I mentioned earlier. The "Classic Footman" model on Reforged (which looks identical to the 2002 model) was recompiled in the Blizzard office. The model itself is the same, but it is "version 1000" and as a result it contains default values for Fresnel Colors and some other stuff in the Materials chunks. None of these values change how the model looks, but they slightly change how the binary is formatted. For reasons like this, as soon as you install Reforged, tools like Mdlvis and Magos:

(1) Will be unable to open your game installation to load in-game texture files, because the game install format has changed
(2) Will most likely be unable to open the standard built-in models from Reforged even if you learn how to extract them -- including the "SD classic" models from Reforged -- because they are encoded in the MDX1000 format.

For any "SD classic" model on Reforged, converting to MDX800 format in my tool is lossless, perfect, should generally work every time (unless there are bugs in my tool with the handling of the old format that have always been there, which I have been gradually fixing for 8 years already). So, this applies to the models such as: Female Arthas, Female Demon Hunter, and such.
Now, I don't know who in their right mind would want to convert MDX1000 to MDX800, but actually I have heard rumors of a lot of people doing this because they don't know the difference and they want to open the files in Mdlvis and Magos and old MDX800 blender plugins and other tools made for the Warcraft 3 MDX format from the year 2002.

Enter the notion of upgrading your models to the new format. My tool is technically capable of making this happen, kind of. But you want to do this in two steps. The first step is to change your model to format 1000. I have a menu option for this. It should be as simple as "Scripts -> Assign FormatVersion 1000" and then saving the file. When you do that, my tool will start writing default values for Fresnel Colors and other information in the new format, but those values will not change how the model looks at all.

Then there is an additional option that I created. It is not as good. I created it during Reforged Beta and have not worked with it too much since then. This second option is "SD -> HD (highly experimental, requires 900 or 1000)". This function was made because during the Reforged Beta, all SD models were dark black as though they were covered in tar, so it was really important for users like you to be able to convert their SD models to have additional HD texture metadata so that their model didn't look all dark on Reforged. And yes, this is referring to both MDX800 and MDX1000 models in the SD style; the Reforged client is indeed able to load old MDX800 model files as-is, courtesy of a large amount of effort by some programmer at Blizzard.

Since the Beta, the Reforged developers fixed all of that and made it so that most old models look fine on Reforged now in general. So I know what you're really looking for is a way to EASILY assign the "Diffuse, ORM, Normal, and Emissive" texture references for your model.

These can easily be assigned from a technical standpoint, but with the development of my tool I have been very lazy and basically just let everyone down with regards to offering this feature. All you have to do to configure these things is to save your model as .MDL extension (during beta I invented my own version of .MDL for Reforged for this tool, so don't try to use the .MDL text models that this program spits out in other Reforged official .MDL format tools). Inside the MDL you can edit anything, even stuff that I forgot to put in my GUI program. In there, the material has a Shader string that can either be blank like the SD classic models, or "Shader_HD_DefaultUnit" which will tell the game this material should be an HD material. Once you tag your material with the Shader "Shader_HD_DefaultUnit", after that the layers declared in the material have a different meaning. The ordering of the layers isn't stuff on top of each other anymore. Instead:

1. The first layer is the Diffuse color, so the Red, Green, Blue and Alpha values of this texture are applied in a pretty standard way similar to old models
2. The second layer is the Normal layer. So, if I understand correctly, Red is X, Green is Y, and Blue is Z. These values define a vector used for per-pixel normal information.
3. The third layer is the ORM layer. (Courtesy of the Hive user Hayate for teaching me the following because I'm an oldschool model editor too) These stand for:
   • Ambient (O)cclusion -- loaded from Red
   • (R)oughness -- loaded from Green
   • (M)etalness -- loaded from Blue
   • Team Color amount per pixel is loaded from the Alpha Channel of this layer.
4. The fourth layer is generally the Emissive layer. Similar to diffuse, the Red, Green, Blue values are again used for a color, but this time it can be thought of as "glowing", I'm sure you can find more tutorials about the meaning of Emissive stuff online, too.
5. The fifth layer is generally the Replaceable 1 team color texture based on looking at in-game examples. And I could be wrong about this, but I'm just looking at the in-game Reforged models to learn how this stuff works. The idea in general would be that based on the alpha channel of the ORM layer, when we decide something should be team color, this color's texture is used to decide *what color that should be*. And they're just using the same old replaceable texture ID system from the old MDX format that this was based on.
6. The sixth layer is generally always a reference to the same texture in every single model, and that texture is "ReplaceableTextures/EnvironmentMap.blp". It's a picture of some trees. I think the idea is that we load this texture as a reflection on shiny metal things, or something like that. I have not spent much time testing with it.

So, I can provide you with this overview of what I see in the data when I look at the models, but again that doesn't actually necessarily help you to have a GUI where you can just push GO and make a Normal Map or an ORM Map for an existing texture. I think a lot of people want that right now, but I have not gone and built such a tool yet, so you will need to find someone else for that or do it yourself. There are a lot of tools for making Normal Maps but I'm not totally sure if they're all encoded the same way. The key idea is that the Normal Map textures are an image file where the Red/Green/Blue pixel data is loaded by the computer and repurposed so that those same number values are used in different ways. I think sometimes you might have a program where "Red" is your Z value coming "out of the page" and other times maybe there would be a program where "Blue" is the Z value coming "out of the page". I have not dived too deeply into the specifics of this on Reforged. I found it easier to take existing Reforged models and combine them into funny things, like porting the Priestess of the Moon to be riding on a Snowy Owl:
Owl Moon Priestess Reforged

Like, do the new normal maps entirely replace the old lighting model?

I have limited experience actually updating old content to Reforged, but from a technical standpoint the old Normals chunk is still present on Geosets and so I think it is still used. I'm not exactly sure how it is used, but I think maybe it is combined with the Normal Map rather than only using one or the other. I suggest doing further independent research on that topic, because I personally don't have solid experience dealing with it yet.

As that ofc would making material work in Margos and Mdlvis impossible.

Yea as soon as you move to Reforged it's the end for Magos unless you want to update his tool. Speaking from having updated my tool, it would probably be easier to actually update Magos to load the new format than it would be to fix my tool to have all the features of Magos. However, last I heard @Magos shows no signs of returning from his 10-year-offline streak.
I downloaded a copy of the "sourcecode" of his tool before his website went offline, but supposedly he deliberately removed essential components so that nobody else could make patches to the software, intending that they would use his code for learning (such as building my own tool) instead. It's kind of a shame, but since he said that, I have never even tried to compile his sourcecode.

How "off" do models without weight painting look in the new renderer? Are there ways to rig models for that, or people with access to 3dsmax able/willing to rig character models for other ppl?
(EDIT: Is it possible to create, rig and animate models in the classic tools and have them be "upgraded" through the official tools? And again, are there people able/willing to do that?)

There are no "official" tools yet. Stop saying that. I am the tools, and I have gone mad with power, so it would be really good if someone replaced me sometime soon. @loktar is looking like a possible candidate with his new converter .

It is true that in the past, vertices formed a group and that group was attached to a bone node. Now, instead, each vertex has a 4-component vector of bone IDs that it is attached to, and 4 weights for each bone ID, so that it can load animation bone information in a weighted manner. When my tool converts back to MDX800 from these models, you lose that extra animation quality. I heard some people were doing it anyway so that they could import these models into Blender. That's crazy and I made the back converter precisely to prove to people that it was a bad idea (although this was criticized for my inability to bake normal maps into SD textures for the backported models, which would in theory make them look better than in my experiment. If it's not clear, that is a video of Reforged assets running on the 2002 game engine).

The rigging in the old format can load up in the new format without issue. Will it look the same style as Reforged content? That's a good question, it might not. Users undertaking this effort have complained that there is a substantially irritating bug with the Reforged client in the HD mode. Any unit who has a binary FaceFX facial animation chunk for talking during the Portrait animations does a lockout on that unit's soundset so that other HD models who do not have their own binary FaceFX blob cannot use the soundset. The manner in which to generate these binary blobs is not currently known to me, so if I make a new model and I want it to talk in HD mode, I have to use the facial animation of an existing unit. My tool does a poor job supporting making a custom model with working FaceFX chunk, although I have done manual tests outside of the tool that showed that it is in theory somewhat possible, such as the moving lips of the lich king hero portrait in this video.

 

[loktar]

As Retera said, his model studio is your best/only choice for upgrading models to v1000.

The version option in my converter basically only changes the version number and (should) write default values for new features. However, it still just writes matrix groups/vertex groups and has no support for creating/converting to skin weights. It also has 0 support for all the newfangles material/texture stuff.

I may develop this further in the future based on Retera's stuff, but it's very low in my priority list. I also have no way of testing as of yet, because I don't have Reforged (yet?).

In short, use Retera's tool. If we're lucky he'll keep updating RMS, but that's of course up to him. My guess is he'll be smashing war and release irregular smaller updates to RMS.
He's also pretty much the community's authority on everything model-related (along with GhostWolf but he's not very active in the community. Also eejin.).

---
Maybe a bit off topic:
I don't think I'll ever make anything that requires actual rendering, because I have no experience with it and I'm not really interested in learning it.
I do, however, have an idea for a non-visual model editor (NoViM Editor or something ) which would basically combine the power of direct MDL notepad editing with a handy GUI and automated changes/warnings where necessary/useful (and without to need to convert models).
If that idea eventually materializes, maybe it could also benefit RMS by allowing it to focus on the other end of the spectrum, but who knows.

 

[Retera]

I have no experience with it

That makes two of us. Just copy Ghostwolf's code. I mean, I guess I have experience now but just look at the tool on the front page of this thread.
Not to derail the thread, but my Reforged support for SKIN chunks and the other stuff all came from copying Ghostwolf's code, too. He figured it out before I did.
I suppose I was just more practical about it, because my code is mostly an editor while his is mostly a viewer.

 

[Xardian]

Okay, first of all, thank you loads for the indepth insight on all of this new stuff!

So just to recap from my pov. Basically, as I am working only with classic tools made for wc3, It would be a massive hit to my workflow, if I would try to upgrade to Reforged now, as I'd need to redo the entire conversion process for every edit I make to my models... But it is definitely possible...

Ofc that means checking, that the map doesnt get bigger than the 256 mb map size limit overall, which would ofc be a massive constraint, considering, I would now need to use 3 more textures per model, in the DDS format, and possibly in 1k resolution instead of 500p.
But should still be possible


But, a few thoughts came to mind as well,

(1) is it even possible to have Classic wc3 and reforged installed alongside oneanother? I mean, I come from a time, where that question would've been laughed at, but nowdays, with all those launchers and self entitled programs doing whatever they want, I am not so sure anymore ^^

(2)

Once you tag your material with the Shader "Shader_HD_DefaultUnit", after that the layers declared in the material have a different meaning. The ordering of the layers isn't stuff on top of each other anymore. Instead:

Sooo... does this mean, that layered textures do not work anymore at all, and the layering system is completely replaced, without an option to assign more than one material layer? Or it has an added aspect of how it should treat each layer separately?

This is a big question to me, because in my map, I am creating a system with replaceable armors, utilising the "mountain giants - warclub" ability to replace a layer on my models texture with the underarmor texture...
So how is this done in reforged? Does the mountain giant just have a replaceable model now? Or just no texture changing at all? Or does it only change the diffuse, without touching the Normal map and the ORM map?

Ofc if this system doesnt work like that anymore in reforged, I would have a hard time being able to port it over, even if I could get the rest of the textures to work.
And it is not that I replace the entire texture, but just a layer over the "nude" texture of my model, so I do need to have the layering working, as the same armor textures are supposed to be used on several different character models.
But in essence, I'd need to have a separate material, that I would be able to exchange in the game, including Diffuse, Normal, and ORM, layered over the base material.

(3)

It is true that in the past, vertices formed a group and that group was attached to a bone node. Now, instead, each vertex has a 4-component vector of bone IDs that it is attached to, and 4 weights for each bone ID, so that it can load animation bone information in a weighted manner. When my tool converts back to MDX800 from these models, you lose that extra animation quality.

So you said, your tool has an animation editor, can your tool also be used to rig for this new format? As then I could ofc use that to create the improved rigs for my models, while using the animations I have already created.


But again a big thank you to you two already, that has helped me a lot with my decision ^^
Gonna stick with classic wc3 for now and get reforged to test around with, as soon as I got the spare change.

In short, use Retera's tool. If we're lucky he'll keep updating RMS, but that's of course up to him. My guess is he'll be smashing war and release irregular smaller updates to RMS.
He's also pretty much the community's authority on everything model-related (along with GhostWolf but he's not very active in the community. Also eejin.).

And big thank you for the contacts and outlook for the future! ^^

I may know a programmer with knowledge in the russian language(s) too, although not sure if he is up for a task like that, he never even played wc3 O:


Cheers, and again thank you!

 

[loktar]

That makes two of us. Just copy Ghostwolf's code. I mean, I guess I have experience now but just look at the tool on the front page of this thread.
Not to derail the thread, but my Reforged support for SKIN chunks and the other stuff all came from copying Ghostwolf's code, too. He figured it out before I did.
I suppose I was just more practical about it, because my code is mostly an editor while his is mostly a viewer.

Yeah I remember Matrix Eater It's come a long way since then.
But yeah, if I were to try a visual model editor, it's be a project spanning years. I'm an amateur programmer as is, always learning as I go, so adding that kind of complexity would take too long and it would never get finished/releasable.
A non-visual one seems feasible for me though. Once my mdlx parser matures a bit more on the editing side, it'll probably be relatively easy.