Blender -> Spark .model exporter

BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
edited January 2016 in Mapping
Update January 23 2016
New, fully featured version is out!
http://forums.unknownworlds.com/discussion/140282/blender-model-compiler

Old version saved below.
Hey guys! At long last, we have a way to export models directly to spark from blender. Before I say anything else, I want to stress that this is a WORK IN PROGRESS. There are many more features planned, but I don't know when I'm going to have the time to work on this, so I figured I might as well release what I've got now.

WHAT IT CAN DO RIGHT NOW:
-Visual geometry
-Collision geometry -- multiple reps! (eg "move" for movement collision, and "default" for bullet hits)
-Re-orient/scale the model upon exporting so it's correct in game.

TODO LIST:
-Animation support -- this is by far the biggest challenge for writing this exporter.
-Create a workflow such that the .blend file can be dropped in the source directory and builder builds the .model file. In other words, adapt it to the existing workflow.
-Create an importer for the .model format. Probably won't be too long on this, already have a rough prototype.

I made/am making this for a few reasons:
1) Tighten up the workflow -- Until now, every model needed: 1 DAE file for visual mesh, 1 DAE file for each collision rep, 1 model_compile file to be run by builder. This exporter eliminates most (eventually all) of that hassle, making the workflow much smoother -- especially when it comes to making small revisions to the model. Currently, you can export directly from blender to a single .model file in one step. Eventually, I plan on adding support for builder to "build" .blend files.
2) Until now, the only way to make models for ns2 was to export to collada and just pray that it liked you enough to maybe work. If you wanted working collision geometry or animations, you had to pirate an old copy of 3dsmax, and use the exporter that came with the game... (still do for animations... :( ) Now, you can do it all with blender.

HOW TO USE:
(I'm putting this here, before the link and the installation instructions because this is a little bit more complicated than just exporting spark geometry)
-Ensure that every mesh you want to be seen in game has a material with a valid texture. It will attempt to find a diffuse texture (a texture mapped to the "Color" channel in blender), but if it doesn't find one, it will go with the first texture it finds. If it doesn't find any textures, it will throw an error.
-Ensure that every mesh you want to be seen in game is a member of a group called "visible" (CASE SENSITIVE!!!).
-Ensure that every object you want to be a collision solid is a member of a group called "physics_*****", where "*****" is the name of the collision rep you want it to be part of. (Eg "physics_move" for movement collision, or "physics_default" for bullet collision). This one is also CASE SENSITIVE.
-OPTIONAL: If you would like to re-orient the scene upon exporting, create an Empty object and call it "origin" (CASE SENSITIVE!!!). What this does is it translates, rotates, and scales the scene such that the 'origin' object is at coordinates (0,0,0), rotation zero'd out, and scale is at 1.0. For example, say you had some map geometry imported and you were converting it to a model, you would create an 'origin' object, and set the scale of it to 39.37 in all dimensions. This means the entire scene is scaled down from 39.37 to 1.0 upon exporting. This also means, if you so desire, you can rotate that origin object so the final, exported model is rotated. Same with translation. NOTE: if all you want to do is convert inches to meters upon exporting, there's an export option for that. This 'origin' feature is mostly intended for re-centering a mesh when exporting.
-Run the export script like you would for any other format.

INSTALLATION:
-Locate your Blender directory. Mine is: 'C:\Program Files\Blender Foundation\Blender'
-There will be a folder inside that is named after the version number of Blender. Mine is 2.72, and please note that this is the version of blender I have programmed this script for. It might work with other versions, it might not...
-Inside, navigate to "scripts" -> "addons", then unzip the three files (__init__.py, ExportSparkModel.py, and SparkIoCommon.py) into this directory. It should then look something like: 'Blender\2.72\scripts\addons\io_scene_spark_model\ExportSparkModel.py'
-Launch Blender, and go to File -> User Preferences. Choose the "Addons" tab, filter by "Import-Export" on the left side, and find "Import-Export: Natural Selection 2 Spark Model Exporter" in the list. Click the checkbox to the right to enable it.
-You should now see "Spark Model (.model)" in the File -> Export menu.

NOTES & COMMON ISSUES:
-Yes, the console spams a crap-ton of info at you when it exports... Just... just take this in lieu of a progress bar.
-Unlike the spark geometry importer/exporter, which defines smooth edges using Blender's "mark sharp" functionality, the model exporter uses the smoothness of the mesh like you'd expect.
-An error will occur if not all visual meshes have UV coordinates.
-Physics objects will automatically be run through a convex hull generator. The exporter does NOT attempt to divide meshes into individual convex pieces. One mesh = One convex solid. To make your collision rep out of multiple collision solids, use multiple objects, not multiple pieces within the same mesh.
-Ensure you've got valid materials on all your visual objects.
-Ensure every face of your mesh references a valid material slot number. For example, if you have two materials on a mesh, with some faces belonging to each, and you delete one of the material slots, the mesh data will still reference the now invalid material slot number. This will cause an error if you try to export with it in this state.
-For each material when exporting, the script will "fix" the texture's file path to reference the output directory. This means it will replace "modelsrc" with "models", and "materialsrc" with "materials", as well as replacing back-slashes with forward-slashes, changing the extension from ".whatever" to ".material", and trimming the entire path so that it starts with "materials" or "models" if it can find those words. If worse comes to worst, you can always manually type in the material's file path into the texture path like so:
SPkpmPy.png
Blender won't like that, and it'll tell you "Can't Load Image" but this at least ensures that the exporter definitely knows the correct path, regardless of whether it displays properly within Blender. Though, you shouldn't need to do this if you've been following the proper naming convention for your assets:
-Props, and all the unique materials and textures for that prop go under "modelsrc/props/[mapname]/[mapname]_prop_name_like_this.psd".).
-Albedo maps do NOT get a suffix (ie "_albedo"); they should be the same name as the material file.
Pro tip: While in mesh edit mode, you can hit spacebar to bring up the search box and find an operation called "Convex Hull". This is the same operation that is run by the export script to ensure the physics meshes are convex hulls. Go ahead and run this ahead of time to get a better idea of what the mesh will look like.
-If you do manage to hit an error when exporting -- don't forget, that means you did something wrong, and that you should feel bad -- then you'll probably have a leftover object called "tempProcessingObject" or something like that. You can safely delete this. Normally the script will delete this for you, but if it's interrupted by an error, it doesn't get destroyed.

Now, with that all out of the way, here's the link to download!
https://www.dropbox.com/s/cc5hhxrdlwda50j/io_scene_spark_model.zip?dl=1

Once again, I want to stress that this is a WORK IN PROGRESS. I had originally planned to only release this once it was "complete", but once I started working on it, I realized how difficult this would be, and how long it will take. Eventually I decided it would be best to at least release what I have so far -- as the vast majority of props in ns2 are static anyways -- and continue working on getting animation implemented.

Here's a short video demonstrating how this could be used to convert level geometry into a prop.

Cheers!
EDIT: Oh and as usual, post any comments/criticism/bug reports here...... well not HERE here... but you know... below...

Changelog:
2-5-15
-Performance improvements:
--Tangent and Bitangent values are now calculated per-triangle, rather than per-vertex. They're the same for every face anyways, now it takes 1/3rd of the time.
--Vertex data is written more efficiently, resulting in a massive reduction of file writing time. Vertex chunk alone took 19 seconds on my test model, now takes 1.

1-24-15
-First release

Comments

  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    Cool,

    Maybe a little video for showing the process of exporting a simple mesh. Not that the text isn't enough, but when it's visual ppl can reproduce it faster/better. It also shows to the non blender expert where to click for "debugging" when something is wrong.
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    edited January 2015
    Cool,

    Maybe a little video for showing the process of exporting a simple mesh. Not that the text isn't enough, but when it's visual ppl can reproduce it faster/better. It also shows to the non blender expert where to click for "debugging" when something is wrong.

    Uploading a quick video tutorial right now. :)

    EDIT: Video is up. :)
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    Good.

    What if i wanted to create an entire new object ? My question is about textures.

    How is it processed right now (copy paste from editor to blender) ? Should it be only using textures known by the editor ?

    How would we proceed in case of complete new textures ? I'm sure you already saw the "model" textures and see what i mean. Would it work with UV Map ?

  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    edited February 2015
    Good.

    What if i wanted to create an entire new object ? My question is about textures.

    How is it processed right now (copy paste from editor to blender) ? Should it be only using textures known by the editor ?

    How would we proceed in case of complete new textures ? I'm sure you already saw the "model" textures and see what i mean. Would it work with UV Map ?

    It works with any textures. All it does it tell it what .material file it needs to look for. You still have to make the .material files yourself. :)

    See, in a model file, the material path is encoded up to the root directory (the 'output' or 'ns2' folder).

    Take for example the "models/props/biodome/biodome_trimmods_01_str1.model" file. Here's what the material chunk looks like in hex.
    mouF8Aj.png

    As you can see, there's only one material in the chunk, and it is "models/props/biodome/biodome_trimmods_01.material". Notice it doesn't care about anything higher than the "models" folder, which conceivably could be either: A) "ns2", B) "output", or C) the name of a published mod.

    When the exporter encodes the .model file from blender, it looks at all the blender materials that are assigned to a mesh, and for each blender material, it tries to find the "best" texture, which is first of all any texture that is assigned to influence "color". If it can't find one of those, it takes any old texture. It then looks at the file path of that texture. It replaces the last instance of "materialsrc" and "modelsrc" with "materials" and "models", respectively. It then strips off everything on the file path in front of "models" or "materials". Finally, it changes the file extension to .material.

    For example, say you've got a PSD file loaded into Blender, it's your diffuse map, and it's a texture of a material that is applied to your object. Let's say the path of this file is
    C:\....blahblahblah...\moddingstuff\awesomemap\mod\source\modelsrc\props\awesomemap\awesomemap_awesome_extractor.psd
    
    This will become:
    models/props/awesomemap/awesomemap_awesome_extractor.material
    

    Or maybe you're like me and your files aren't so nice and neat.
    What I've done sometimes is add another texture slot on top of the other textures, I name it "export_path" (name doesn't matter, just helpful). Then I load any random image, doesn't matter, and manually type in the path I want it to use in the "source" box. It won't load a texture into blender, but it'll still display properly, assuming you have GLSL viewport enabled. This ensures the exporter gets the right path.
    20LX5Dy.png

    When you use the geometry importer, it's creating all the materials for you, so it should work immediately with the model exporter.

    Not sure what you mean when you're asking about UVs, but that's the ONLY way models can be textured in spark. In fact, it's required. Non-textured models will not export.
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    Thanks for the update. I just wanted to make sure i have the right information.

    Why?
    I may start a little project which would be some kind of props library.
    We have at disposal many types of things but there's always something missing. Example : pipes. Some sets have the 45° turn some others don't. So the idea is to make a complete set (Straight, 90°, 45°, T, Y, cross sections, broken, etc.) of pipes and provide different textures for multiplying the number fast with minimal effort.

    It can be done with many other things. Rocks, stones, pillars and such. So better make sure the "industrialized" method will work.
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    Thanks for the update. I just wanted to make sure i have the right information.

    Why?
    I may start a little project which would be some kind of props library.
    We have at disposal many types of things but there's always something missing. Example : pipes. Some sets have the 45° turn some others don't. So the idea is to make a complete set (Straight, 90°, 45°, T, Y, cross sections, broken, etc.) of pipes and provide different textures for multiplying the number fast with minimal effort.

    It can be done with many other things. Rocks, stones, pillars and such. So better make sure the "industrialized" method will work.

    That's awesome! Do you mean you're going to be modifying existing props to add missing components, or are you making your own props from scratch? :)
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    edited February 2015
    Just released an update that drastically improves performance... up to 60% faster in some cases!!! One model that took about 19 seconds to export now only takes about 5. :D
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    BeigeAlert wrote: »
    That's awesome! Do you mean you're going to be modifying existing props to add missing components, or are you making your own props from scratch? :)

    I have to train myself to blender again. It has changed since i left it ages ago.
    The easiest is to get all from scratch. At least the simple things like pipes.
    Having all sources and processing it at will is more convenient.
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    BeigeAlert wrote: »
    That's awesome! Do you mean you're going to be modifying existing props to add missing components, or are you making your own props from scratch? :)

    I have to train myself to blender again. It has changed since i left it ages ago.
    The easiest is to get all from scratch. At least the simple things like pipes.
    Having all sources and processing it at will is more convenient.

    I've got a rudimentary .model importer working for Blender if you'd like that. You'll need to tweak some things in the script, as it's just a packaged text file, not a nice addon like the exporter is. (ie you'll have to change a string variable to the path of the model you want to import).
  • ArchieArchie Antarctica Join Date: 2006-09-19 Member: 58028Members, Constellation, Reinforced - Supporter, WC 2013 - Supporter
    Thanks for this mate, you've been a very large help. Do you think you could petition the NS2 team to maybe give us the updated assets? i mean it's a given that they gave us access to the older assets, but why not include the new ones? it's not like they don't want them to be shared.

    It's not hindering the CDT's development and it can only be good as a stimulant, i mean i would love to get working with Animation, just holding out for your progress, it's a weekly ritual to be honest! <3
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    nizb0ag wrote: »
    Thanks for this mate, you've been a very large help. Do you think you could petition the NS2 team to maybe give us the updated assets? i mean it's a given that they gave us access to the older assets, but why not include the new ones? it's not like they don't want them to be shared.

    It's not hindering the CDT's development and it can only be good as a stimulant, i mean i would love to get working with Animation, just holding out for your progress, it's a weekly ritual to be honest! <3

    Animation support is coming soon (VERY soon) with the Blender exporter. The way it'll work is you'll run your .blend files through builder just like any other format, and it'll generate the model file from that.
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    Hi,

    I tried with Blender 2.76. I'm stuck with texture apparently.
    I was not importing from Spark editor
    I did a simple object from scratch and applied a texture (blender render type).
    Traceback (most recent call last):
    File "C:\Program Files\Blender Foundation\Blender\2.76\scripts\addons\io_scene_spark_model\__init__.py", line 77, in execute
    return ExportSparkModel.save(self, context, **keywords)
    File "C:\Program Files\Blender Foundation\Blender\2.76\scripts\addons\io_scene_spark_model\ExportSparkModel.py", line 327, in save
    tri.material = AddMaterial( tempObj.material_slots[t.material_index].material , bMats , sMats )
    File "C:\Program Files\Blender Foundation\Blender\2.76\scripts\addons\io_scene_spark_model\ExportSparkModel.py", line 188, in AddMaterial
    path = '.'.join(path.split('.')[:-1]) + '.material'
    AttributeError: 'list' object has no attribute 'split'

    location: <unknown location>:-1

    Could you test it and tell what is wrong ?

    BTW can we use Cycles instead of Blender Render ? It's clearly easier to setup textures and mapping in Cycles (nodes). I did not try to be sure to stay close to the example (video).

    The goal is to open Blender; do the stuff and export without any help from the editor (copy / paste).

    thx
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    Hi,

    I tried with Blender 2.76. I'm stuck with texture apparently.
    I was not importing from Spark editor
    I did a simple object from scratch and applied a texture (blender render type).
    Traceback (most recent call last):
    File "C:\Program Files\Blender Foundation\Blender\2.76\scripts\addons\io_scene_spark_model\__init__.py", line 77, in execute
    return ExportSparkModel.save(self, context, **keywords)
    File "C:\Program Files\Blender Foundation\Blender\2.76\scripts\addons\io_scene_spark_model\ExportSparkModel.py", line 327, in save
    tri.material = AddMaterial( tempObj.material_slots[t.material_index].material , bMats , sMats )
    File "C:\Program Files\Blender Foundation\Blender\2.76\scripts\addons\io_scene_spark_model\ExportSparkModel.py", line 188, in AddMaterial
    path = '.'.join(path.split('.')[:-1]) + '.material'
    AttributeError: 'list' object has no attribute 'split'

    location: <unknown location>:-1

    Could you test it and tell what is wrong ?

    BTW can we use Cycles instead of Blender Render ? It's clearly easier to setup textures and mapping in Cycles (nodes). I did not try to be sure to stay close to the example (video).

    The goal is to open Blender; do the stuff and export without any help from the editor (copy / paste).

    thx

    Yea I'll take a look at it. Unfortunately, no on cycles materials. The node system kinda throws a wrench in the works. Really, all the exporter needs to see in order for the .material to get hooked up right is have the very first texture slot set to "diffuse", and manually set the file path to the relative material path.
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    @UncleCrunch

    Okay the bug is because with the textures that have been assigned to the model, there is no way for the exporter to know what the material name should be in the model. What it's looking for is something along the lines of "modelsrc/props/mapname/mapname_my_awesome_prop.psd". Packing the texture into the .blend doesn't work.

    Here's what I recommend for the workflow:
    -Create a new texture slot at the top of the stack, and un-check the checkbox next to it so it isn't visible.
    -Click "new" to create a new image texture assigned to the slot.
    -Click "ok" to dismiss the generate image dialog.
    -Change the "source" of this new image from "Generated" to "Single Image".
    -A blank file path box appears below. Enter the NS2 RELATIVE path to your .material file in here
    -For example: "models/props/mymap/mymap_myprop_01.material"
    -Ensure that "Diffuse Color" is checked under the "influence" section, otherwise the exporter will probably skip over this texture in favor of another.

    That should do it.

    One other thing, I noticed you had setup a physics object for this prop, but you didn't put it into the proper group -- it was still in the "visual" group. It needs to be in "physics_default" for it to work.
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    Yeah i know about the file. It's a dummy file which is lighter than the original. I may have been too fast. I packed the texture for the example.

    I'm afraid i couldn't get it to work. But i admit i suck in "blender render". Cycles is more my cup of tea.

    There is another thing.
    Basically the path is a declaration for Spark engine to tell it where to look when processing the prop. But it's kind of getting in the way when working. It's like you need the final material before the object. If you want to test many versions of the same object it tends to get messy. You have to save the texture and then export, test, etc...

    It would be more simple to give to the exporter a set of paths info (config panel, or before saving file) to be able to forge the files. 'files' as x.model & x.material. So the exporter have all paths info needed (even for a mod... especially for a mod) to forge the files.
    Up to the user to copy the files in the right directory (if not saved in there already).

    I'm telling that because if you want to make many objects and export them in a industrialized manner, it becomes a hassle to set up all this things without error. including an add-on config setup will simplify things in a large scale and remove potential errors. Usually you work on one mod, maybe 2. But not changing every 5 minutes.

    Something like :
    (in Config panel) 'material dir = //material/MyModNameBacauseITidyMyRoom/'
    The exporter have to find the texture filename (it's already there when painting the object is the user isn't dumb) and assemble both strings. Let's say the user name the texture identically to the .material filename (with or without the .material).

    -> //material/MyModNameBacauseITidyMyRoom/texture.material.
    save .model
    save .material (forged and saved if possible)

    Why saving both ? So we can test many versions in a blink of an eye by just changing 2 or 3 parameters in the blender file.
    • It may be more simple for the exporter (but i let you be the judge of that).
    • We would probably be able to use any rendering engine in Blender as the exporter only needs a filename which is in UV/image editor. As the UV/Image stack is available by any script/rendering engine in Blender... You may have a shot.
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    edited October 2015
    But you see, the problem with that approach is it sacrifices the flexibility of being able to use any material from any directory; it locks the user into only using materials from a single directory. For example, if you are doing multiple variations of the same model (see derelict's plants), you want to be able to re-use existing materials not necessarily from the same folder location (the plants use the kodiak tree bark material, which is NOT in the derelict folder).

    The exporter doesn't make the .material files for you, that's your responsibility. This exporter was designed to simplify the workflow of exporting models to spark, not to completely eliminate careful planning and a proper art creation workflow.

    Here's the workflow I used when making the derelict stuff and, more recently, the cardboard box on Temple.
    1) Get modeling out of the way -- including high-poly models/sculpts to bake normal map from.
    2) Bake normal map (I actually have a spark-specific addon for this coming soon, so you can bake MUCH more accurate normal maps).
    3) Create textures in photoshop and blender.
    4) Load up all textures in a Blender material so that we get a rough idea of what it's going to look like in the game engine. (And trust me, you should be using Blender Render for this, as it's much closer to how a game engine will render the mesh than the un-biased, super realistic methods cycles uses).
    5) Setup physics models.
    6) Export to Spark, and view in model viewer. Make necessary changes to material textures in photoshop, and reload model in spark, without needing to re-export model.

    Also, heads up, there will be soon™ a solution that allows you to work with .blend files as source files through builder, so you won't need to go through the hassle of exporting individual files, it'll be exported for you just like the old model workflow, except of course it's all nicely contained to a single .blend file, even for multiple variations. Also, animation export is coming. :)
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    Ok I got it to work.

    But some things are in the way of a great and fluid workflow.

    The exporter is quite ok for any single prop as a statue or a vehicle... name it. But if you try to make a set of props there can be some obstacle there.

    Suggestion 1 : The scaling gap problem
    0buulnzgvzba.jpg
    In this picture the bloc is a square of length 64 units in Blender. When exported it looks like this. Resizing it makes it ok. But the thing is if you have several bloc with different shape and details it tends to look like this.
    Even if we scale it to the next multiple of 16.vkasp8j46x7l.jpg
    We shouldn't see the line in the middle. only the 2 large lines.
    I think the 39.37 factor may need more digits after the mark. If it's the right number there must be something wrong somewhere.
    NB: unit = none for these props.

    Suggestion 2 : Material source image variable, is it really necessary ?
    The Spark model format (though I didn't explore it in detail) isn't including any bitmap in it, but only :
    • 3D objects
    • Uvs datas
    • reference to the material as a path variable.
    I mean it's quite clear this is the minimum data to include in it.

    The exporter uses the "source" variable from the material texture (blender render) which is attached to the object (in the tree structure). So basically we only need a variable to retrieve and export to get the same results.

    What about using the "custom variable" we can attach to the object itself ?
    like so
    3unuq4t65nio.jpg
    10 seconds to set up.
    (don't mind the object name)
    I'm not the specialist but I think it would be an easier procedure to find the variable (especially if you know its name) instead of following the tree down to the source variable (through the material etc.).

    We would be able to :
    • use any render engine we want (even the Pixar one), to make the object and texturing.
    • work with many objects and only one material in case of a "theme set" (many props, same style or texture). We wouldn't need to set variables in many materials which basically are the same. Only 1 variable attached to the each object.
    • work with an entire set in the same file without being subject to mistakes like "I thought I was on the A object material and I was on B when exporting" etc.
    • Make complex material that can allow us to work on multiple texture (specular, emissive etc...) at the same time in the same Blend file (yay).
    With this 2 things I would be able to complete and maintain
    • The many pipe prop mod I wanted to do
    • making more props for my map
    • making other props to enrich the whole thing
    • giving to the community tutorials to create from scratch props in Blender and how to work fast and easy. In English (though mine is poor) and French (ouais la baguette et le beret, merde quoi!).

    So... now you understand that you can't do anything else in order to save the world but to deliver this. Ho... And I expect this on my desk for tomorrow morning... and maybe I can forgive you... if I'm in a good mood... Bring coffee too... ... 2 sugars...
    ;)

    On a side note:
    I experience some strange things with the viewer. It looks like it has a cache. If you load A then B and then you reload A (but a supposed new version) it stays on the A old version. I have to close the application and open it again, then it works. Anyone confirms ?

    I would love to have a reload function in the viewer. Also if it could keep the last config (like physic window open and render physic mesh etc...).
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    Hmm... I hadn't seen the scaling problem before. I'll check it out.

    No, like I said, it ONLY includes a reference to a .material file. You can specify this PER BLENDER MATERIAL, by making a dummy texture that doesn't have any visual impact in the blender viewport, but yet contains the path to the .material file that you want it to use. See, it's designed to "guess" what the name of the .material should be, BUT you can override this, as I explained in my last post.

    I'm not adding per-object materials, because that removes the flexibility of having multiple materials per object, and introduces unnecessary complexity and ambiguity.

    If you want a "theme set" that has many objects using the same materials... use the same BLENDER material with these export settings already configured. ;) The idea here is that 1 blender material == 1 spark material, though the exporter DOES do a pass to ensure that materials are only exported once, to protect against duplicates.

    I think I mentioned earlier as well that this tool will soon be replaced by a new exporter that will also support animations. In addition, this exporter will have the capability of exporting multiple .model files from a single .blend file. So that should help out your workflow if you're creating a "set" of props. You could then, theoretically, work on your entire set within a single .blend file, and when it's run through builder (yes, .blend files will soon be able to be run through builder! :D) it'll generate several .model files.
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    "No, like I said, it ONLY includes a reference to a .material file. "
    Hmmm that's what i said.
    Read instead "reference to the *spark material as a path variable pointing at the filename".

    What I'm saying:
    If there is no material data in the .model file (except a spark .material filename); what is the point of using Blender Render materials "source image" variable ? Blender Render is soon to be deprecated if not already going to oblivion. Also it's a poor solution concerning materials compared to today standards. I got back 10 years in the past re-using it.

    Any engine supporting node style is by far more powerful and allows to work with all the textures we want at once. Ex: I could have a Blender (Cycles) material with many mix sliders that allows me to see 50/50% specular and albedo or 70/30% emmisive and albedo, etc. All this at the same time at the tip of some sliders. I can even simulate a glossy effect as an equivalent to reflection probe in Spark and use a slider to reveal it at will. I name it : Ultimate. + GPU rendering while working... yum yum.

    We can agree it's for working in Blender only and has no influence on Spark rendering. But "If you see better during the session; you perform better". Trust me.

    Ok it will be replaced and builder.exe will process .blend files. But if the artists can't get out of the Blender Render way of doing things, it may be laborious to complete some work. Not to say; I know my way around in Blender and I can cope with it. Maybe it's not that easy for others. Cycles (or other rendering engine that supports node style) is by far easy to learn / work with. And will be the standard in Blender if not already, as the devs go fast implementing things, so fast.

  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    "No, like I said, it ONLY includes a reference to a .material file. "
    Hmmm that's what i said.
    Read instead "reference to the *spark material as a path variable pointing at the filename".

    What I'm saying:
    If there is no material data in the .model file (except a spark .material filename); what is the point of using Blender Render materials "source image" variable ? Blender Render is soon to be deprecated if not already going to oblivion. Also it's a poor solution concerning materials compared to today standards. I got back 10 years in the past re-using it.

    Any engine supporting node style is by far more powerful and allows to work with all the textures we want at once. Ex: I could have a Blender (Cycles) material with many mix sliders that allows me to see 50/50% specular and albedo or 70/30% emmisive and albedo, etc. All this at the same time at the tip of some sliders. I can even simulate a glossy effect as an equivalent to reflection probe in Spark and use a slider to reveal it at will. I name it : Ultimate. + GPU rendering while working... yum yum.

    We can agree it's for working in Blender only and has no influence on Spark rendering. But "If you see better during the session; you perform better". Trust me.

    Ok it will be replaced and builder.exe will process .blend files. But if the artists can't get out of the Blender Render way of doing things, it may be laborious to complete some work. Not to say; I know my way around in Blender and I can cope with it. Maybe it's not that easy for others. Cycles (or other rendering engine that supports node style) is by far easy to learn / work with. And will be the standard in Blender if not already, as the devs go fast implementing things, so fast.

    First of all, a game engine's renderer is nowhere comparable to the unbiased rendering system of Blender Cycles. Second, you should NOT be previewing materials in Blender, as Spark renders things very differently to Blender's renderer (cycles or not). If you want to make adjustments to your material, you should be making these adjustments in photoshop, or whatever texture painting program you're using, and view the changes in the Spark Viewer. If you're using Blender to paint your textures... that's great... but there is nothing you or I can do to make that an accurate representation of how it will appear in Spark. Third, for the purposes of making an asset for the Spark engine, there is nothing blender or cycles's node-based material setup can offer. If the day comes that Blender removes the regular material stack in favor of nodes, then I will be happy to re-write the material system so it uses nodes. In the mean time, you are more than welcome to pick up python on your own and make these changes yourself, and I will be happy to review them and merge them in if they are satisfactory.

    Also, as you yourself point out "the devs go fast implementing things, so fast.", is all the more reason for me to NOT attempt to write a script for something that is changing rapidly, but rather to prefer the old method that has been untouched for so long. I've got other things to work on besides updating this every time the devs decide to change/break something.
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    edited October 2015
    The thing is with what i suggest you wont even have to deal with a material stack...


    Edit:
    That should do it
    import bpy
    import random

    ObjectRandName = random.randrange(1,10000)
    InitialVarValue = "Yolooooooo!!!"
    TargetVarValue = "\materials\mymod\mymodel.material"

    #Fresh new object/mesh
    print ('
    ')
    print ('Hi!')
    bpy.ops.mesh.primitive_cube_add(location=(0,0,0))
    ob = bpy.context.active_object
    ob.name = "Yolo" + str(ObjectRandName)



    #start like if I know the object name.
    CurrentObj = bpy.data.objects[ob.name]
    CurrentObj.select = True

    print ("New object is : ", CurrentObj.name)

    #setting custom property to test it later
    bpy.context.object["SparkMaterialFilename"] = InitialVarValue
    CustomVar = bpy.data.objects[CurrentObj.name].get('SparkMaterialFilename')
    print ("Custom Var named SparkMaterialFilename = " , CustomVar)

    bpy.context.object["SparkMaterialFilename"] = TargetVarValue
    CustomVar = bpy.data.objects[CurrentObj.name].get('SparkMaterialFilename')
    print ("Custom Var named SparkMaterialFilename = " , CustomVar)

    print ('')
    print ('')


    instead of "def AddMaterial(bMat, bMats, sMats):" in ExportSparkModel.py

    Procedure would be to :
    • Set an initial value for spark material filename
    • check if the custom Var exists,
    • Yes -> get it, replace an initial value like "dev.material"
    • No -> log it
    • Save it
    Even if the artist mess up, it has a default dev.material filename (any other official .material would be ok).

    This can do just fine and is more simple to manage for artists as for script writers.

    File included
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    edited November 2015
    @UncleCrunch

    Tell you what I'll do. For the next iteration of the exporter, I'll add support to where it'll look for a custom value in the material, and if it finds one, it will use that, otherwise it will fall back on the old method. It'll look for the material's custom value, not the object's. See, by putting the custom value on the object, you're limiting yourself to one material only, but if the custom value resides in the material, this is actually exactly what we want. My only complaint is the box isn't quite long enough to see the entire path without having to do that awkward click and drag to scroll thing... but that can't really be helped.

    To be honest, I actually didn't know custom values could hold strings. I thought they were decimal numbers only. This is great news indeed!
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    BeigeAlert wrote: »
    @UncleCrunch

    Tell you what I'll do. For the next iteration of the exporter, I'll add support to where it'll look for a custom value in the material, and if it finds one, it will use that, otherwise it will fall back on the old method. It'll look for the material's custom value, not the object's. See, by putting the custom value on the object, you're limiting yourself to one material only, but if the custom value resides in the material, this is actually exactly what we want. My only complaint is the box isn't quite long enough to see the entire path without having to do that awkward click and drag to scroll thing... but that can't really be helped.

    To be honest, I actually didn't know custom values could hold strings. I thought they were decimal numbers only. This is great news indeed!
    This is more due to the props system and it's not really a limitation as whatever the system you use; you have to change a value ( or assign material B ) to export 2 objects using 2 different textures (industrial goal). Say a beam and a rusty version. Even if it's easy you have to do it.

    The real issue lies in the props system. The 3D exported data embarks the reference to the .material file.

    Actual Prop.model
    Rely on 1 .model file containing the 3D data and ONE reference to a .material file.
    3D file:
    • 3D data
    • UV data
    • reference to .material file

    Heaven of props system for Prop.model
    Rely on 2 files. 1 for the 3D data and 1 for the definitions (A text file made of references).
    .prop file
    • reference to 3D model.3Dmodel
    • reference to .material file
    .model file
    • 3D data
    • UV data
    You can even go further by adding a definition for the collision model or a specific shader... oooh too Hot !


    It would need to change how the engine do things props at loading time but it's not gonna be a killer.
    • I guess the final build image is totally scripted / automated (don't even answer "no"; it will lower the esteem level i have for UWE down to -1000).
    • Minor development as many things are already relying on definition files (like .material)
    • Manage the change for mods. A mail to modders +forum announcement will do. Can't do much more.

    A system like this allows several things.
    -Any model can use several textures. Clean, burnt, rusty etc... without using more 3D data than it really needs. 1 model multiplied by X textures.
    -If the 3D artist is clever and smart he can create objectS that has UV using the same areas. So you can create X 3D models and multiply by Y possible textures. Beams, grates, rocks are best candidates. Ex: 3 .model x 5 .material... gives 15 possible .prop file. I think you sense a disturbance in the force now.

    Meditate.

    Let's get back to business.
    Next version: When is it scheduled ?
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
    The 3d data is never the problem. The textures and animations take up by far the most space in the model file, and animations can be re-used.
  • UncleCrunchUncleCrunch Mayonnaise land Join Date: 2005-02-16 Member: 41365Members, Reinforced - Onos
    First props, then animation if possible. But no1 priority is props.
    Improvements like i describe can come later.
  • BeigeAlertBeigeAlert Texas Join Date: 2013-08-08 Member: 186657Members, Super Administrators, Forum Admins, NS2 Developer, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Diamond, Reinforced - Shadow, Subnautica Playtester, Pistachionauts
  • FlaterectomyFlaterectomy Netherlandistan Join Date: 2005-02-03 Member: 39643Members, NS2 Playtester, Squad Five Blue, Squad Five Silver, NS2 Map Tester, Reinforced - Shadow, WC 2013 - Shadow, Subnautica Playtester, NS2 Community Developer, Pistachionauts
    Well tuck it back in before people notice.
Sign In or Register to comment.