X4 Wiki
Space shortcuts
Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

HOWTO - Hacking X4 Foundations

 

Communications

EGOSOFT Forums for all languages and games

Mods

Hacking References

forum](https://forum.egosoft.com/viewforum.php?f=192)

Tools

  • X Catalog tool 1.11 - .CAT unpacker. Requires you register your game with the EGOSOFT site to download. There's also

XERCES, which doesn't require registration, and has clicky.

code completion for x4 variables and instructions for using

+ Those who do know emacs and want code completion [might find joy
      here](http://lgfang.github.io/mynotes/emacs/emacs-xml.html) but
      you will have to reprocess the
      XSD's. [This](https://martinfowler.com/articles/emacs-nxml-completion.html)
      could be of use too, typing that extra `="` can be a pain.

XMLpatch

XR modding guide, which is similar enough to help.

Publishing on Steam

Gameplay References

 

 

 

 

 

 

Most of the code and data that governs the game is located in XML files. Mods can add to the game data, or even alter the stock data as its loaded.

Debug Logging

To enable the games logging features, you'll need to specify some parameters at game launch. the "Properties" in steam has a "Set Launch Options" button that lets you enter them there, or you can create shortcuts that launch the game through steam with different options.

To invoke X4 with options from a desktop shortcut, set its target to "C:\Program Files (x86)\Steam\Steam.exe" -applaunch 392160 -skipintro -debug all -logfile debuglog.txt -scriptlogfiles

  • -applaunch 392160 - tells steam which game to launch. The rest of the options get sent on to that game, in this case, X4.

  • -skipintro - just what you'd think

  • -debug all - turns on the debug log, and sets the log filter to "all". Other filter levels are error, general, scripts, scripts_verbose, economy_verbose, combat, and savegame. See the XSD entry for <debug_text>

  • -logfile debuglog.txt - specifies the name of the file the debug log is written to. This will be in the My Documents/Egosoft/X4/[digits]/ directory, beside your saves/.

  • -scriptlogfiles - turns on script logging, see the <debug_to_file> reference in coommon.xsd

    • -showfps - again, as you'd think

Unpacking game files

Euclid's forum thread Getting Started: Tools Scripting and Modding talks about unpacking the files. The more recent official tool or XERCES are options too. X4FProjector might also be of use.

.CAT and .DAT and mods

Packing your mod into this format is not required to use it in game, or publish for others to use. It can offer advantages and is required for distributing your mod on Steam Workshop, but can be handled later by the WorkshopTool.

The readme file with the XRCatTool says (in part): txt Notes for modders on catalog usage in X Rebirth, X Rebirth VR Edition and X4: Foundations:

At first, the game load catalogs of the form ##.cat (## = 01..99). Loading stops at the first missing file.

In each enabled extension, the game then tries to load the following files: - subst##.cat (starting with subst01.cat) - substv###.cat (### = current game version, e.g. substv130.cat) - ext##.cat (starting with ext01.cat) - extv###.cat (### = current game version, e.g. extv130.cat)

The files in the ext_* catalogs are interpreted as part of the extension. This is the recommended way of altering the game, it allows for a high degree of compatibility between mods.

The files in the subst_* catalogs are interpreted as relative to the game root (adding/replacing base game files). This should only be used if there is no other way to adjust the game assets. Please make sure that players are aware of the potential incompatibilities with other mods.

File Structure: What are we looking at?

XXXX restucture

There's 8GB of data here, its quite a lot to go through. Three fourths of it is graphics and audio, and fortunately for us, everything is reasonably organized and the things we're likely to be interested in are informatively named.

I've unpacked into directories corresponding to the .DAT files, so my paths all begin with digits right now. This was a mistake, but as so often happens it can be an educational one. I will correct it later.

The game overlays the contents of each .DAT it loads and builds a view of the sum of all of them. In that view the 01/, 02/, etc directories don't exist, nor does the root directory of a mod being loaded.

The files in a mod get overlaid as their own DAT, even if they're not packed. My QuickBuilder mod consists of two files: quickbuilder/content.xml quickbuilder/libraries/wares.xml

The content.xml tells the game about the mod; and libraries/wares.xml gets merged into the game's stock wares.xml file when the mod is loaded.

You can replace an existing ware completely this way, or you can use XMLpatch to modify existing values. Adding new content doesn't require <diff>.

The 01/assets/ and 05/assets/ also demonstrate the file overlaying. 05/assets/ redefines some ships previously defined in 01/assets/, example:

diff -u0 v2.50-unpack/01/assets/units/size_l/macros/ship_arg_l_destroyer_01_a_macro.xml v2.50-unpack/05/assets/units/size_l/macros/ship_arg_l_destroyer_01_a_macro.xml 
--- v2.50-unpack/01/assets/units/size_l/macros/ship_arg_l_destroyer_01_a_macro.xml	2019-05-28 19:19:27.975130500 -0500
+++ v2.50-unpack/05/assets/units/size_l/macros/ship_arg_l_destroyer_01_a_macro.xml	2019-05-28 19:38:01.144485000 -0500
@@ -2 +2 @@
-<!--Exported by: Michael (192.168.3.81) at 21.11.2018_18-11-06-->
+<!--Exported by: nick (192.168.3.120) at 10.05.2019_16-52-31-->
@@ -23 +23 @@
-      <people capacity="36" />
+      <people capacity="44" />
@@ -45,0 +46,3 @@
+      </connection>
+      <connection ref="con_shipstorage_xs_01">
+        <macro ref="shipstorage_gen_xs_01_macro" connection="object" />

To create a unified, unpacked filesytem that looks like what the game sees, I've used the command: for i in 01 02 03 04 05 06 07 08 09 ; do cp -Rvf $i/* ../v.250-unpack2/ ; done

and that results in the unpacked structure to use for reference in your own work. The only reason to unpack the .DAT files individually like I did is if you're a terminally curious person like me and want to see what gets overwritten.

However you get there, you should end up with a directory that looks like this:

aiscripts
assets
cutscenes
index
jobeditor.html
libraries
maps
md
music
particles
scriptproperties.html
sfx
shadergl
t
textures
ui
voice-L044
voice-L049
vulkan

also see the XR Wiki File Structure page

[12:35 AM] h2odragon: @UniTrader  im trying to reference the vanilla assets by using <source geometry="assets\structures\connectionmodules\struct_arg_cross_01_data"/> in my component file. But from a mod that path wouldn't be correct?
[12:36 AM] UniTrader: it is correct 
12:36 AM] UniTrader: all file paths are relative to the X4 exe
[12:37 AM] UniTrader: this includes cat/dat archived files which are virtually extracted in the game root (01-09), in mod folder (ext_01) or again in the game root (subst_01)

The XSD files

First thing to look at is libraries/*.xsd, and scriptproperties.html. These files document the various program operations and variables you see used in the rest of the XML files. You want to keep them in mind.

The XML

The Mission Director guide section on script structure has a good explaination of how an MD script is built and is a clear example of how this is all wrapped in XML.

Gooey, chunky, XML, which when coupled with the "do what i meant" development ecosystems favored today can be offputting to those of us who are accustomed to text editors. Or code and data specifications having different syntaxes. But it works, and its truly not so bad. Better than working with a hex editor.

Internationalization, or "why {1041,10146}"?

The files in t/ contain translations of strings used in the game. The files contain <page> sections, each of wich is full of <t> tags. The two numbers in the identifier refer to <page id="X"> and <t id="X">.

For example, in aiscripts/order.trade.middleman.xml we see "{1041,10146}". xml

<param name="warebasket" required="true" default="if this.ship.job then this.ship.warebasket.list else null" type="list" text="{1041, 10146}" comment="Wares"> <input_param name="type" value="'ware'"/>

The context makes it obvious, but: searching t/0001-l044.xml (which is the English language texts) we find: xml

Wares

The same method will be very useful for other less obvious strings.

True Names

Many people are bewildered at this point by the game's internal names for everything. Specifically, where to you find them and how do you know them.

You'll sort of sop things up organically eventually but thats slow and painful. Therefore I've put together lists of some of the more important concordances.

All the wares in libraries/wares.xml have anid attribute, which is the name by which you reference that ware in code. They have a tags attribute too, like container, mineable, modules, ship, equipment, inventory, research, or many others.

Ship Connections are important when dealing with ships in any detail.

XXXX Macros

Having a copy of "all the macros in one file" might be useful; here's a couple of bash commands to get that. In the unpacked game files directory: bash find assets/ -name "*_macro.xml" -print > macrofiles for i in `cat macrofiles` ; do echo -e "\n\n$i\n" ; cat $i ; done > allmacros.xml

The find command lists all the files under assets/ whose names contain "_macro.xml", and puts that list into a file named macrofiles. The for command reads the list of file names, then for each file prints out a header and the contents of the file; and all of that output is wrapped up into allmacros.txt.

find assets/ -name "*.xml" -and -not -name "*_macro.xml" -print > cptfiles

will get component file names.

What did you want to change?

Where you go from here depends on what you want to change. With the game data files on hand you can now begin downloading and unpacking (when necessary) other people's mods to look at. Do that. Do a lot of that.

Missions happen in md/, ship commands are governed by aiscripts/, wares are defined in librares/wares.xml. assets/ contains the definitions of all the ships, stations, rocks, etc.

XXXX Where to put your mod /how to develop

filenames: all lowercase can avoid problems

forum thread

gamestarts

The 'New Game' menu is made up of nodes found in libraries/gamestarts.xml. This has it's own XSD.

Creating your own gamestart involves adding new <gamestart> nodes by defining them in your mod's libraries/gamestarts.xml. See apricotstart for a template to begin from.

The md/ApricotStartSetup.xml file contains the mission director cues that do other things at the game's beginning. the id attribute of the <gamestart> node determines which _Init cue is called.

In the game files the setup is done in md/Setup_Gamestarts.xml, which contains more code for test starts than is accessible from the vanilla game. Here's a copy of the gamestarts.xml that goes with those. original link

The XR Custom gamestart howto might be useful too.

Laying out a ship in libraries/gamestarts.xml may require looking up the connection names for elements you wish to include (the path attribute). I've created table of ship connections for starts for that. It lists only the connections for engines, weapons, turrets, and shields, which you need in the <ship> section.

aiscripts

aiscripts/order.mining.routine.basic.xml is simple example of an default behavoir aka infinite ship order. The non-inifinte orders and subroutine scripts are different, more later. XXXX

Inside the <aiscript> tag is an <order> tag; (see the libraries/aiscripts.xsd for details of the attributes).

The <params> block defines parameters which may be presented to the user when they issue this order on one of their ships. Other more complicated orders will have <init>, <interrupts>, and may have <attention> nodes other than unknown.

<actions>; finally, some action! This is where your code begins to influence what a ship does. Execution arrives at this point + when the order is issued + after an interrupt has been handled + after the order script has completed or used <return />

Which means that a ship has to figure out what its doing after every gate transition, for example, fresh and without assuming anything about its current state.

A script can create and order with <create_order> (yet another example of reasonably named stuff). If the order is immediate, it will begin running right now, but your script will continue executing after the <create_order> node as well. cleanup, if needed, and exit (jump to finish) at that point and wait for the new order you created to complete, after which your default behavior will resume executing.

XXXX or possibly you could wait and periodically check this.ship.order?

Or you can run another aiscript/ with <run_script> (as is done in aiscripts/order.mining.routine.basic.xml) which behaves like a traditional subroutine.

A script referenced with <run_script> doesn't need the <order> tag in its structure.

md

X Rebirth Mission Director Guide

MD scripts are used for other things, like the FlyBy Looting mod; which sets up a cue to fire every 10 seconds and check for loot around the ship.

ships and stations

BrummBear's Building your own ship for dummies

weapons

weapons and turrets are defined in assets/props/WeaponSystems/*

bullets are defined in assets/fx/weaponFx/.

@mewosmith tip: Values in weapon macros (damage etc) using min and max attributes for randomness will also require a value attribute. This is apparently the only place value is required and not optional.

How the ecell plant is defined in vanilla

  • wares.xml - defines the blueprint ware
    • <ware id="module_gen_prod_energycells_01"
    • includes <component ref="prod_gen_energycells_macro" - aka 'the macro'
  • index/macros.xml - maps macro name to macro node
<entry name="prod_gen_energycells_macro" 
 	   value="assets\structures\production\macros\prod_gen_energycells_macro" />
  • index/components.xml - maps component name to model data
<entry name="prod_gen_energycells" value="assets\structures\production\prod_gen_energycells"/>
  • assets/structures/production/macros/prod_gen_energycells_macro.xml -

this station's macro definition xml <macro name="prod_gen_energycells_macro" class="production"> <component ref="prod_gen_energycells" />

  • assets/structures/production/prod_gen_energycells.xml - defines

connections and references model xml <component name="prod_gen_energycells" class="production"> <source geometry="assets\structures\production\prod_gen_energycells_data"/>

  • assets/structures/production/prod_gen_energycells_data/ - actual compiled model data
    • *.xmf including assets_structures_production_prod_gen_energycells-[collision,lod0].xmf

maps

enenra's universe creation page

DeadAirGate Overhaul

DeadAir's Sector map with IDs - has ID numbers of vannila clusters plus DAG's new sectors.

effects

(2019-07-05) mewosmith's effectnames.xml - Test harness and notes on included effects

characters

Appearance is governed in libraries/character_macros.xml Helmets can be removed there...

ui / Lua

  • [SirNukes Named Pipes

API](https://github.com/bvbohnen/X4NamedPipes_API) - Truly epic, early implementation of named pipes to MD. Data I/O to a python script running beside the game. I'm still thinking up uses for this one.

G_Workaround forum thread

From Discord (20190624): > Mysterial: @mewosmith @UniTrader if you're trying to make a &gtnew menu by adding your own ui.xml like XR then it won't work. The &gtmod-friendly path is bugged. You have to replace one of Egosoft's &gtui.xml files. That's what the whole _G workaround thing is about.

EGO Jira Bug report

Event list: https://gist.github.com/NodusCursorius/56f55f267a5f0f5509b6f46c6a1d3703 VTable list: https://gist.github.com/NodusCursorius/ac6bd44080ebee47662204c1ed983dcb Lua function list: https://gist.github.com/NodusCursorius/b61f26177fdcb490d2456e353d57f363 API misc list; https://gist.github.com/NodusCursorius/c8a97cc73a03fe6bdb466863e1a8ef84

Ventures with mods

from the Discord:

[2019-07-01 11:15 PM] Nodus Cursorius: X4: Foundations - 2.5 Prevent modified flag (yay ventures with mods) X4.exe Offset: 0x112140 - 3 byte change 48 83 EC 28 E8 27 5B 57 00 85 C0 0F 95 C0 48 83 C4 28 C3 ; pre-hack 0F 95 C0 48 83 EC 28 E8 27 5B 57 00 85 C0 B0 00 90 48 83 C4 28 C3 ; post-hack B0 00 90 ^ ^ ^

; origial
IsGameModified  proc near
                sub     rsp, 28h
                call    sub_140688870
                test    eax, eax
                setnz   al
                add     rsp, 28h
                retn
IsGameModified  endp

; changed
IsGameModified  proc near
                sub     rsp, 28h
                call    sub_140688870
                test    eax, eax
                mov     al, 0
                nop
                add     rsp, 28h
                retn
IsGameModified  endp
// original
bool IsGameModified()
{
  return sub_140688870() != 0;
}


// changed
bool IsGameModified()
{
  sub_140688870();
  return 0;
}
  • No labels