Displaying Custom Maps in-game: Tutorial and Japan/Korea Screenshot

[JC_Denton]

I couldn't fall asleep, so I decided to see if I could get a new map displaying in-game.  Visually, that is.  I've been working on a Europe map since I hadn't known how to import new map images when I started (otherwise I likely would have done North America, since I live in North America, and it would be fun to play on a map with even more cities I've traveled to).

The good news is, I have some results!  The bad news is I'm still figuring out exactly how I did it.

Let's see if uploading an image works here...




  screenshot_2022-9-10-238-58.png (Size: 2.23 MB / Downloads: 834)

I started out with Japan and Korea because it's a smaller area than the U.S.  Figured that meant I was less likely to run into crashes due to running out of memory or something like that.

Steps (please note these are not yet final):

1.  Download the full-res Blue Marble image from Wikipedia (https://en.wikipedia.org/wiki/File:Blue_Marble_2002.png).  Make sure to right click -> Save Link as on "Original File".  It may well crash your browser if you try to view it in the browser.

2.  Open it in a program that can handle a 500 MB PNG file.  I found that GIMP and IrfanView (64-bit) were the two I had installed that could handle it, but IrfanView was much faster.  IrfanView also lets you drag a box around a subset of the image (I'll use India as the example for this post), and adjust it with the bounding rectangle in case you accidentally selected too little or too much.  Neat!  Once you've got the area you want, paste it into a new file and save it as a PNG in its own folder, optionally shrinking it along the way (not necessary for Japan at 3495x2799, but might be for larger maps).  Let's say you named the PNG BlueMarble_India.png, it should wind up as BlueMarble_India.dds.


3.  Open Aorta.exe (which live in Gear City/Mod Tools), go to Batch Convert, choose the folder with your Blue Marble PNG, and choose a fresh folder to put the converted DDS file into.  Make sure Save as DDS is checked.  Click "Batch Convert..." and uncheck all the options, then click Okay.  It might take a couple seconds depending on the size of the map selection.

4.  Go to the GearCity/media/Maps folder, and copy the Europe Test folder, name it after your map, e.g. India (GearCity/media/Maps/India will be the map folder).

5.  Go to GearCity/Mod Tools and open GearCity_ModToolSuite.exe.  Choose Map Data Tools in the upper-left, and open GearCity/media/Maps/India/MapFileForModTools.xml.  Customize the Map Name and Map Description, let's say the Map Name is "India Map".  Leave the rest the same, including the Map Picture being "mapPicture_EuropeTest.dds".  I had a bunch of trouble with crashes due to not finding the map picture so I decided to leave well enough alone.

6.  Save the MapFileForModTools.xml file.  Then click Export Finished Map Files.  Export it to a fresh folder, let's say GearCity/media/Maps/India Exported.  It will export it to a sub-folder based on the name you gave it, such as GearCity/media/Maps/India Exported/India Map.  Copy the "India Map" folder to the GearCity/media/Maps folder.

7.  Go into your "India Map" folder, and then its MapArt folder, and make a copy of the MapArt.zip file, and then extract it.  Rename the "EuropeCityMap.material" file to "IndiaMap.material", open it, and on line 14 after "texture", change it to BlueMarble_India.dds.  Save the file.  Then find the BlueMarble_India.dds file from step 2, and place it in this same MapArt folder.  Compress both the DDS and the IndiaMap.material file into a new "MapArt.zip" file that will live in this folder.

8.  Within your "India Map" folder, open MapFileForGame.xml, and on line 17 ("ChangeMap"), change EuropeCityMap to IndiaMap.  Eventually you'll also want to change the lat/lon markers so cities display properly, but at this point you'll still have the cities in Europe and they'll just display weirdly like in the picture above.  Getting a fresh slate and proper cities is for a future tutorial.

9.  Fire up GearCity (with mods disabled), and click New Game, and click on "India Map".  It might crash here, if so go to "GearCity/GearCity/Logfile.log" and see what it says.  If not, click through to start a game and hopefully you see your map!  If it crashes, check that log file.  Most likely it failed to find a file somewhere.

(Note to self: I created this with "Japan Two" and copied the exported version to "Japan Test II".  The earlier "Japan" version is hopelessly broken for some reason, and probably needs a diff tool to figure out why.  The working version is also backed up)

-----

Also, some of the changes around the .material file are probably intended to be done via "Edit Map Artwork Loading Years" in step 5 (the "Map Data Tools" in the Mod Tools).  I haven't tried or verified that, but it might make things slightly easier.

----

Hopefully this is of use to someone.  I'm writing it up in large part for my future self, when I try that North America map.  I'm also fine with it being repurposed for the Wiki's page on the Map Data Tool.

I should also note that at this point I have no plans to create a Japan/Korea map in the short/medium term future. I'm currently finishing version 2 of my Europe map, with North America, and then version 3 of Europe likely next. Japan/Korea might be after that. In part because I've had reasonably good luck finding demographic/economic data for Europe (especially Spain/Portugal/France/Italy), and am confident I can find good info for at least the U.S., but am not sure if I can for Japan and Korea, in part due to the language barrier. But also just not knowing what's available... English is my native tongue, but it was way easier to find Spanish data than English for various historical reasons, primarily that there are about five types of counties in England.

So if anyone wants to start a Japan/Korea map... it's available.

[Eric.B]

Are those cities GPS coordinates for cities in Japan and Korea?

When you export the map there is a tab or something for regional maps. In that tab you will need to put the GPS coordinate for the four corners of the map that are visible in the game. After that, the game will place the cities where they are in relation to the map based on GPS.

Japan actually has a pretty big following in this game. It's now our 3rd biggest non-English speaking country (Behind Germany and Brazil). I would imagine this map would be immensely popular with them. If you make the base, I can probably point a few of them here to pick up on it.

[JC_Denton]

Those aren't Japanese/Korean cities, but the ones in Europe from Europe test. I now realize that why they are where they are is because I haven't modified the GPS coordinates of the map, and because the dimensions (including aspect ratio) are different than Europe Test. I figured out the coordinates of my map selection by roughly lining it up on Google maps sateillite view and wrote them down, so I should be able to add that.

At first I was trying to get actual Japanese and Korean cities from the Base City Map, but there seems to be an issue in the City Editor where if you link it to a TurnEvents file and remove a city, it crashes (it works with AI files that are linked). So I didn't have a good way to scrub city references from TurnEvents, and decided to focus on the unknown part which is the map. A side goal is to have "blank" city/AI files, and no-cities-linked TurnEvents, to make it easier to start from scratch in new regions. I can probably scrub the TurnEvents programmatically pretty quickly to create that but it was 3 AM.

Good to know that the game is popular in Japan, I wouldn't have known! Yeah, I can probably package it up as a "base map with a handful of cities to start with" in the near future.

[JC_Denton]

For anyone interested in developing this farther, you can download it from https://ajtjp.com/software/GearCity/Down...ing%29.zip (97 MB). After decompressing it, put the "Japan Test II" folder in your "GearCity/media/Maps" folder, and it should show up in-game.

It definitely needs a city revamp, and I'm pretty sure the .zip can be slimmed down a bit, but it should get anyone who just needs the base map up and running.

I am unlikely to get back around to this until 2024 or later, so if anyone wants to create their own Japan/Korea map/scenario, please go ahead!

[JC_Denton]

I've been playing around with this some more.  The current results:



At this point I've figured out that the reason cities were displaying in the wrong place is that I hadn't set the latitudes/longitudes correctly.  I've also figured out that I can use ../ notation to pull in the Base City Map (or Europe Test) scripts/logos/news art/etc., which simplifies setup a bit and will reduce map size when those assets are being re-used.  In the Japan + Korea map, when pulling in base cities, they're still in the wrong spots but they're just too far north.  Most likely that means my latitude coordinates are off, though I haven't verified the city latitudes in the scripts, either.  I suspect part of it is also simply the hazards of a flat map of a spherical world.

As you can see on the new map, the cities show up pretty much in the right areas.  It also took less time to create this one since I had these notes and had done it once before.

Another lesson learned was avoid spaces in the map name (the one inside the ChangeMap element, and used for the name of and name within the "material" file).  Is this 100% required?  I am not sure, but after I simplified the naming and eliminated spaces, the map stopped being completely white in-game and started showing up.  I had avoided spaces for Japan and tried not to for the Great Lakes, but alas, it seems best to avoid them.

I've also confirmed steps 5 and 6 are not necessary, as long as you have MapFileForGame.xml configured right it will work.  It might be nice to have the ForModTools version to experiment with some of the options there, but as a software developer who is used to working with pointy XML files, I tend to find saving a couple steps to be more valuable.  I'm thinking of putting together an improved "minimal viable map instructions".  The key remaining piece is starting with a blank slate city file, and blank news script in terms of city interactions.

Though I still have two questions:

 - Does anyone visit the forum?  It seems very quiet.  Does anyone who does visit have interest in building maps, if the instructions were sufficiently easy to follow?  To me, GearCity seems like a natural fit for having multiple maps with different levels of granularity, but it's admittedly been a bit difficult to figure out how to create a new one from scratch.
 - I noticed that the coordinates are top-left, bottom-right, etc. lat/long, rather than just left/right/top/bottom.  Is the game sophisticated enough that I could put in a "diagonal" rectangle rather than a "traditional" one and it would work?  E.g. could I do a map that goes southeast/northwest along the length of Italy, and put in those corner coordinates, and it would still put cities in the right place?  I don't have a good way of generating those maps yet but it's sort of cool if it works.

[Eric.B]

 - Does anyone visit the forum?  It seems very quiet. 

Not so much anymore. The Steam forums killed most activity here. That's why I closed the forums in 2018/2019-ish. I only reopened it for the FBS, but it has underperformed, and some contributors haven't even made accounts yet.

I'm hoping for an uptick in activity once we hit the first milestone. Hopefully, the paywall attracts more users here. That said, the chat (discord/IRC) sees a little more activity. At one point, it was pretty much dead too.

That said, your Japanese map did gain a few tweets from our Japanese twitter community. GOG users also only have access to maps/mods on the forums. Sadly there aren't too many of them, but they exist.

Does anyone who does visit have interest in building maps, if the instructions were sufficiently easy to follow?  To me, GearCity seems like a natural fit for having multiple maps with different levels of granularity,

I've seen a few people making maps over the years, adding and removing cities. But none have taken it as far as you have with changing the map texture as far as I know.

but it's admittedly been a bit difficult to figure out how to create a new one from scratch.

I had planned on making a better video tutorial for the mod tools, including making a map from scratch. But the manual is taking longer than expected, especially now that I only touch GearCity Documentation on Mondays.

I suspect part of it is also simply the hazards of a flat map of a spherical world
....
 - I noticed that the coordinates are top-left, bottom-right, etc. lat/long, rather than just left/right/top/bottom.  Is the game sophisticated enough that I could put in a "diagonal" rectangle rather than a "traditional" one and it would work?

No. The maps need to be in WGS84 or equirectangular projection for the coordinates to work. Both standards are rectangular, so the map will have to be rectangular, thus why we need only 2 coordinates. From the looks of it, you're using NASA's blue marble maps, which should be the correct projection. So with your cities being slightly off, your map coordinates are off. I suggest finding exact reference points you can naturally see to clip your map atm else than trial and error until the cities align.

E.g. could I do a map that goes southeast/northwest along the length of Italy, and put in those corner coordinates, and it would still put cities in the right place?  I don't have a good way of generating those maps yet but it's sort of cool if it works.

Out of the box, no. But, so long as the map is a rectangle, you can put the map in the game. Then you manually convert city coordinates for the map rotation. For instance, you could set your Italian map corners as 0,0, and 1,1. In real life, it would be 46,13, and 35,7. If you're rotating it 90-degrees it would become 13,46 and 7,35. Rome at 41,12 would become 12,41. Which then converted between a range of 0 to 1 would be 0.164, 0.545. That should put Rome on the map where it should be. However, this would screw up shipping, so you will need to tweak the numbers somewhat so they're closer to the real values.

Alternatively, the game does support 3d modeled maps. If you have those talents, you can do any type of map you want. Classic Map and both Territory maps are 3d modeled maps. The exported entry names for the city markers have to be the number corresponding to the city's ID. That and the lack of camera or model rotation is the only restrictions.


By the way, if you attach images to the post, and then inline them here on the post, they should appear inline now.

[jerrod247]

After reading this thread I was inspired to make a attempt at getting an map into the game. The information here has been helpful. I am using a fantasy map. Here are my current results:
   

[Eric.B]

After reading this thread I was inspired to make a attempt at getting an map into the game. The information here has been helpful. I am using a fantasy map. Here are my current results:

I'm detecting a hint of Azgaar! It'll be quite interesting converting the data points there into a territory map.

[jerrod247]

After reading this thread I was inspired to make a attempt at getting an map into the game. The information here has been helpful. I am using a fantasy map. Here are my current results:

I'm detecting a hint of Azgaar! It'll be quite interesting converting the data points there into a territory map.

It is in fact an Azgarr map. It seems to work well in this case because the generator creates latitude and longitude coordinates for all of the cells as well as the cities on the map so location on the map should in theory be easier to nail down.

[JC_Denton]

I am glad to see someone else is looking into custom maps too! And that writing down what I learned proved helpful for someone other than myself.

I was thinking today about when I might like to update my Europe custom map to incorporate the new ability to separate industry and infrastructure growth, and which map I want to play my first "real" (not just seeing the mechanics) game of 2nd Gear. But there aren't a whole lot of options for maps to choose from yet. So having more of them would be a good thing!

I'll also be curious what your experience is with adding cities. I've tried a few different approaches over the years, starting with the built-in tools, then moving to the XML files, then writing some tools of my own that better matched how I had my data organized. And that itself has evolved over time. But I'm also looking at it from a historical data standpoint, so I don't know what would work best on a fantasy map.

[jerrod247]

I was thinking today about when I might like to update my Europe custom map to incorporate the new ability to separate industry and infrastructure growth, and which map I want to play my first "real" (not just seeing the mechanics) game of 2nd Gear.  But there aren't a whole lot of options for maps to choose from yet.  So having more of them would be a good thing!

This will eventually very interesting to play with, the map I am currently working on is for original version (which I suppose is 1st gear) as it seems to be more stable at the moment and I am still trying to figure out how to implement things.

I'll also be curious what your experience is with adding cities.  I've tried a few different approaches over the years, starting with the built-in tools, then moving to the XML files, then writing some tools of my own that better matched how I had my data organized.  And that itself has evolved over time.  But I'm also looking at it from a historical data standpoint, so I don't know what would work best on a fantasy map.

So far when adding cities I have mainly used the built in tools and modified xml files as needed. On the front of keeping my information organized I made a spreadsheet with the data in it as I find it easier to keep track of in this way. Eventually I might have to follow your lead on making a tool, as I would really love to make it so data could be input via something like a CSV file. 

I am curious how you come up with values for variables without a straight forward real world value such as resource rate or manufacturing skill.

[JC_Denton]

I was thinking today about when I might like to update my Europe custom map to incorporate the new ability to separate industry and infrastructure growth, and which map I want to play my first "real" (not just seeing the mechanics) game of 2nd Gear.  But there aren't a whole lot of options for maps to choose from yet.  So having more of them would be a good thing!

This will eventually very interesting to play with, the map I am currently working on is for original version (which I suppose is 1st gear) as it seems to be more stable at the moment and I am still trying to figure out how to implement things.

I'll also be curious what your experience is with adding cities.  I've tried a few different approaches over the years, starting with the built-in tools, then moving to the XML files, then writing some tools of my own that better matched how I had my data organized.  And that itself has evolved over time.  But I'm also looking at it from a historical data standpoint, so I don't know what would work best on a fantasy map.

So far when adding cities I have mainly used the built in tools and modified xml files as needed. On the front of keeping my information organized I made a spreadsheet with the data in it as I find it easier to keep track of in this way. Eventually I might have to follow your lead on making a tool, as I would really love to make it so data could be input via something like a CSV file. 

I am curious how you come up with values for variables without a straight forward real world value such as resource rate or manufacturing skill.

Hey, sorry I missed your post, I'm used to different forum software and hadn't seen the alert.

I'm also still targeting 1st Gear for now, as I figure out what all has changed.

For figuring out values... it's a lot of research, and some approximations. Which are complicated when multiple countries are involved because they measure things differently and inconsistently, especially the farther back in time you go. Add in changing borders, and it becomes worse.

But for population, it's kind of gone like this:

- Find stats for a regional population. Idaho. France. The Upper Peninsula of Michigan.
- Determine which localities I want to have in-game. Boise. Bordeaux. Sault Saint Marie. Find stats for their populations.
- Divide regional population proportionally.

This isn't perfect as in most countries, multiple municipalities represent an urbanized area. That definition of "urbanized area" differs by country, and I've found several false positives. "Metropolitan Statistical Areas" in the U.S. are not great because they are county-based and include significant non-urban areas. "Functional Urban Areas" in the EU are worse, because they represent the areas that _could_ commute to a city, which tends to be quite wide.

For resource rate/manufacturing, etc., in 1st Gear I've used "urbanization" as a proxy. So let's say I have Boise, Pocatello, and Couer d'Alene as my cities in Idaho, and they represent 20% of the population in 1900 (not necessarily accurate numbers). In Massachusetts, Boston, Springfield, and Worcester are my three cities and they make up 60% of the population. Massachusetts is more urbanized.

I'd like to include more variables such as availability of key materials such as steel; finding data on it and then incorporating it is what takes time, especially if I want the data to be accurate, not just, "Pittsburgh has a lot of steel".

In 2nd Gear, one of my main goals is to incorporate historical info on road build-out for infrastructure. I've found this at a per-country level for autobahns in the EU, and expect I should be able to find it on the Interstate system in the U.S.

---

GNP is the other big one I focused on initially. Finding country-level information is not too difficult, although in earlier years it's often in the local currency, so I had to find historical currency conversion rates and build some currency-conversion tables (sometimes it required more than one hop to convert). Finding regional info can vary by country. Last year I found an academic publication where researchers from a lot of countries had put together their work to come up with approximate GDPs for all regions in Europe for every decade in the 1900s (think of Brittany or Normandy as the size of the region). That's fantastic for the scale of Gear City, so I'd like to improve my GDP info with their research. IIRC they also did some currency normalization, so if I'd found it earlier it would have saved some work.

I should probably compile all my notes and put them in one place; I mentioned some of this in my EEII thread but quite a bit of the nitty-gritty remains in local text files and archived PDFs.