Hello TR,
I thought it would be finally time to step forth and ask for your opinion on a few things regarding resource-sharing and establishing more consistency between TR and the projects hosted at project-tamriel.
At the moment we are discussing a common Tamriel-Data for S:HOTN and P:C that will either by dependency or by merging include all of the objects in TR_Data.
You can read up on the respective discussion here:
http://project-tamriel.com/viewtopic.php?f=67&t=451
In extension, I thought. it could make sense to speak about a common set of general design directions and lore with TR. Our mods are designed to fit into the same worldspace with Tamriel Rebuilt, so it would only make sense to make it fit together in all other respect.
As there has been some resource-exchange with TR in the recent time, I hope that TR is interested in making our projects consistent with each other, too. In particular, I would like to promote some of our Cyrodiil-lore, as I believe it might hold up to TR's high standards. You can read up on some of it in the following links (I hope, Infragris will not choke me for this):
http://project-tamriel.com/viewtopic.php?f=144&t=318
http://project-tamriel.com/viewtopic.php?f=144&t=194
http://project-tamriel.com/viewtopic.php?f=144&t=329
http://project-tamriel.com/viewtopic.php?f=11&t=382
http://project-tamriel.com/viewforum.php?f=142
Also, much of the internal lore established at TR, will probably have implications on the other provinces. If we know about these things, we can stick to them, over there aswell. This can go from meta stuff all the way down to trivial thingsl like plants. For example, I remember Sload militating against the Nirn Root as a plant? If TR's version of Tamriel, for whatever reason, omits things like these, we should do the same thing over there, of course.
Lastly, we have updated some of the imperial-related assets from TR_Data, like the Colovian Bread and the wines, which is also something that TR might be interested in. At the very least, we could try to have a set of common trade goods that are consistent across the provinces.
Some cross-project coordination
Moderator: Lead Developers
-
Anonytroll
- Website Administrator
- Posts: 135
- Joined: Fri Aug 12, 2005 5:23 pm
Re: Some cross-project coordination
Sounds like an excellent suggestion, consolidating the divergent asset files.Worsas wrote:Hello TR
The issues about differences in poly count and graphical detail are quite real. How about consolidating vanilla-style versions into this common data bsa and keeping a "HQ Patch" or "Official MGSO-alike" on the side?
After speaking to some people on IRC, I was surprised about the general amount of interest in cross-coordination and sharing of assets, and from what I have gathered, most people I spoke to were even interested in a common Data with the projects at PT.
This is far beyond what I initially had in mind. I will still have to gather agreement of the people on the PT board on that and it would be good to still hear the unheard voices on the matter, before work on a common data is started.
I should also say that the new Data planned (but not even fully agreed on) for PT is not merely supposed to unifiy the objects of P:C and SHOTN, but to clean them from duplicates and misfits at the same time. I even thought about going through each asset piece by piece and collect a general highres material library at the same time for future use and more visual consistency.
Another aspect about this all is the yet-unfinished master plan of TR. It would make sense to finish this master plan, before a common data is created. The new data should no longer contain stuff that will be removed later on.
Originally my idea was to develop this data at the side over some months as an ideal data with only stuff that we (I) really want and the way we (I) want it.
But now that people have shown interest in helping at this and now that I think about it more thoroughly, it should be a common process anyway and nothing that happens behind the scene by my deliberate judgement. We should do this forum-based with input by all involved parties.
That is when there is a general consensus on doing this all.
This is far beyond what I initially had in mind. I will still have to gather agreement of the people on the PT board on that and it would be good to still hear the unheard voices on the matter, before work on a common data is started.
I should also say that the new Data planned (but not even fully agreed on) for PT is not merely supposed to unifiy the objects of P:C and SHOTN, but to clean them from duplicates and misfits at the same time. I even thought about going through each asset piece by piece and collect a general highres material library at the same time for future use and more visual consistency.
Another aspect about this all is the yet-unfinished master plan of TR. It would make sense to finish this master plan, before a common data is created. The new data should no longer contain stuff that will be removed later on.
Originally my idea was to develop this data at the side over some months as an ideal data with only stuff that we (I) really want and the way we (I) want it.
But now that people have shown interest in helping at this and now that I think about it more thoroughly, it should be a common process anyway and nothing that happens behind the scene by my deliberate judgement. We should do this forum-based with input by all involved parties.
That is when there is a general consensus on doing this all.
Well, on the TR_Data side, I know that I and Seneca37 are going through and weeding and fixing up our models and textures. So, since we're already cleaning things, it might make sense. I'd be up for it, especially as there would be some common goods such as misc items, clothes, ingredients, etc that would make it across the borders.
On the other hand, Morrowind as a province is fairly unique with its objects, so there wouldn't be too much overlap. That's a good thing.
Also, OpenMW will eventually be released, which means potential new life for this type of project.
On the other hand, Morrowind as a province is fairly unique with its objects, so there wouldn't be too much overlap. That's a good thing.
Also, OpenMW will eventually be released, which means potential new life for this type of project.
Reading through the Project Tamriel discussion and this thread:
I am in favour of aiming towards a unified data library between projects.
I'm not sure whether it might prove a good idea to have secondary data files for the projects containing province-specific data or not; TR_Data includes stuff like regions, scripts and dialogue that would be of no real value to the other projects. On the other hand, in the long-term, we will likely want to merge that stuff with TR_Mainland anyway, so it may just be a temporary issue.
The different visuals of the projects are certainly a concern, but on second thought perhaps not a very large one, in that it won't be any worse than the current situation: when we borrow models from other projects, we provide low-res textures for them anyway. As such, at worst, no different from the current situation, the alternative high-res or low-res textures can just be created and added as needed. There has been some discussion on our end of eventually providing high-res alternatives to TR's textures as well, which should then make it easier for you folks to use our stuff. It's only in cases of armour and clothing that use better bodies where we'd need to do more extensive work, if we wanted to use those designs in our project. But, again, that's no different from the current situation.
Of course, the fact that either the high-res or low-res stuff would probably need to be a separate patch is rather clunky. Perhaps some elegant solution could be found, for example users downloading the data file would be able to choose whether to download a low-res or high-res bsa.
I'd certainly like to see us work together more on the lore front as well. Consistency will only become more important as the projects work towards the Velothi Mountains.
I am in favour of aiming towards a unified data library between projects.
I'm not sure whether it might prove a good idea to have secondary data files for the projects containing province-specific data or not; TR_Data includes stuff like regions, scripts and dialogue that would be of no real value to the other projects. On the other hand, in the long-term, we will likely want to merge that stuff with TR_Mainland anyway, so it may just be a temporary issue.
The different visuals of the projects are certainly a concern, but on second thought perhaps not a very large one, in that it won't be any worse than the current situation: when we borrow models from other projects, we provide low-res textures for them anyway. As such, at worst, no different from the current situation, the alternative high-res or low-res textures can just be created and added as needed. There has been some discussion on our end of eventually providing high-res alternatives to TR's textures as well, which should then make it easier for you folks to use our stuff. It's only in cases of armour and clothing that use better bodies where we'd need to do more extensive work, if we wanted to use those designs in our project. But, again, that's no different from the current situation.
Of course, the fact that either the high-res or low-res stuff would probably need to be a separate patch is rather clunky. Perhaps some elegant solution could be found, for example users downloading the data file would be able to choose whether to download a low-res or high-res bsa.
I'd certainly like to see us work together more on the lore front as well. Consistency will only become more important as the projects work towards the Velothi Mountains.
Hello again,
What do you think are the chances for a common Data.esm until the end of the year?
My plan for the common data files, so far, is as follows:
- We create a completely new data.esm and add the objects from all three projects to it piece by piece.
- The assets themselves remain in the bsa-files of the respective projects, so it remains traceable what came from where.
- Assets that exist as duplicates across the projects, are only added once and are taken from the originating project. This would mean that the assets of the barrow set would be taken from Skyrim_Data.bsa rather than TR_Data.bsa. The other way around any assets that were first in TR_Data (even, if taken from modder resources) would be taken from TR_Data.bsa.
- The new data uses a project-agnostic id-pattern for everything but npcs, groundtextures and scripts.
- We maintain a large text-file with a mapping of the old object ids to their new counterparts. This file would ideally be used by a program that automatically patches all content of the three projects to utilize the new data.esm.
The main thing I hope for, when moving most objects to a different, project-agnostic naming pattern is that the purpose of an object within the gameworld (opposed to its ownership aspect) gets the main attention. All three projects have objects of relevance for the other projects and grouping them into thematical sets would allow to have objects of characteristical similarity in a single place. Simple things like waterlayers or ropes exist in great variety between the three data files. Stuff like this could get bundled into single sets.
With regards to this id-pattern, I see two interesting possibilities:
1. Using the naming pattern I came up with here: http://project-tamriel.com/viewtopic.php?f=67&t=451
The large advantage of that naming pattern is that CS - work on stuff like dwemer ruins or other environments with a homogenous set of objects will be much much more convenient. At the moment you are forced to scroll between the furn- and the in-stuff of a single set. This wouldn't be the case anymore.
2. Using the vanilla naming pattern for all objects and omit the grand prefix. The advantage would be that relevant vanilla objects would end up in neighbourship of similar TR_Data or PC_Data or Skyrim_Data - objects. While you would still need to scroll between furn and in, you wouldn't need to scroll between frequently-needed vanilla - and project objects anymore. Certain objects would be more difficult to find. But it would essentially be a single large naming pattern for everything in the game and would be a good expression of the intend of presenting the game and our addons to it as a unified whole.
It probably sounds like an extremely ambitious and difficult undertaking, but it would be a good chance to re-evaluate everything we have. Deal with misfitting itemstats or other stuff that isn't quite right.
What do you think are the chances for a common Data.esm until the end of the year?
My plan for the common data files, so far, is as follows:
- We create a completely new data.esm and add the objects from all three projects to it piece by piece.
- The assets themselves remain in the bsa-files of the respective projects, so it remains traceable what came from where.
- Assets that exist as duplicates across the projects, are only added once and are taken from the originating project. This would mean that the assets of the barrow set would be taken from Skyrim_Data.bsa rather than TR_Data.bsa. The other way around any assets that were first in TR_Data (even, if taken from modder resources) would be taken from TR_Data.bsa.
- The new data uses a project-agnostic id-pattern for everything but npcs, groundtextures and scripts.
- We maintain a large text-file with a mapping of the old object ids to their new counterparts. This file would ideally be used by a program that automatically patches all content of the three projects to utilize the new data.esm.
The main thing I hope for, when moving most objects to a different, project-agnostic naming pattern is that the purpose of an object within the gameworld (opposed to its ownership aspect) gets the main attention. All three projects have objects of relevance for the other projects and grouping them into thematical sets would allow to have objects of characteristical similarity in a single place. Simple things like waterlayers or ropes exist in great variety between the three data files. Stuff like this could get bundled into single sets.
With regards to this id-pattern, I see two interesting possibilities:
1. Using the naming pattern I came up with here: http://project-tamriel.com/viewtopic.php?f=67&t=451
The large advantage of that naming pattern is that CS - work on stuff like dwemer ruins or other environments with a homogenous set of objects will be much much more convenient. At the moment you are forced to scroll between the furn- and the in-stuff of a single set. This wouldn't be the case anymore.
2. Using the vanilla naming pattern for all objects and omit the grand prefix. The advantage would be that relevant vanilla objects would end up in neighbourship of similar TR_Data or PC_Data or Skyrim_Data - objects. While you would still need to scroll between furn and in, you wouldn't need to scroll between frequently-needed vanilla - and project objects anymore. Certain objects would be more difficult to find. But it would essentially be a single large naming pattern for everything in the game and would be a good expression of the intend of presenting the game and our addons to it as a unified whole.
It probably sounds like an extremely ambitious and difficult undertaking, but it would be a good chance to re-evaluate everything we have. Deal with misfitting itemstats or other stuff that isn't quite right.
I cant speak for anyone here, but as I read this, I must say that one single month (plus christmas!!) is far not enough time for a project that extensive.
But I am one of those who see a unified tamriel in one single mod one day! All hail the emperor
Oh, or better, whole Nirn! Or no, Nirn and every plane of oblivion!! Or no, wait, whole Nirn, Oblivion and the ruins of Battlespire!!! Or that all plus Aetherius? ;p
But I am one of those who see a unified tamriel in one single mod one day! All hail the emperor
Oh, or better, whole Nirn! Or no, Nirn and every plane of oblivion!! Or no, wait, whole Nirn, Oblivion and the ruins of Battlespire!!! Or that all plus Aetherius? ;p
Why cant I simply use an Elder scroll to make my Clocks go slower?
Nah, we are just speaking about a common data files. I'm not expecting to finish a province in a month. 
(looking at my registration date would quickly dissolve such considerations)
And yeah, I think it is possible. At least, if we are only speaking about inserting a large amount of objects into a new esm file and setting id and values.
However, if we want to perform some kind of data re-evaluation at the same time, it will probably not work that quickly.
I would, nevertheless, like to start creating these new data files with just the objects from Skyrim_Data for now. There are many issues with those data files, which I could address when transferring everything piece by piece into a new data.
But it will be necessary to agree on a common id-scheme at first, if it's supposed to be our common thing.
(looking at my registration date would quickly dissolve such considerations)And yeah, I think it is possible. At least, if we are only speaking about inserting a large amount of objects into a new esm file and setting id and values.
However, if we want to perform some kind of data re-evaluation at the same time, it will probably not work that quickly.
I would, nevertheless, like to start creating these new data files with just the objects from Skyrim_Data for now. There are many issues with those data files, which I could address when transferring everything piece by piece into a new data.
But it will be necessary to agree on a common id-scheme at first, if it's supposed to be our common thing.
Just to be clear, I'm completely on board with the idea, but am less familiar with the technical issues of implementing it on TR's end than others in the project, most notably Seneca, who I understand you've already been in contact with. Certainly, I don't recall anyone outright disagreeing with the idea of a unified data file; all of the comments in this thread and that I recall reading in other places such as on IRC have been supportive of the idea. But practically speaking it does indeed look like this will not be possible this year, which is unfortunate.
The focus in this thread has mainly been on unifying files, but how is the sharing of assets? I ask because I got to thinking about Uld Vraech and how close it is to the border with Skyrim. I was curious how many of the assets used in creation of the various barrows and glaciers and flora and buildings are custom to TR versus vanilla, and how many had been shared with Skyrim: Home of the Nords. If all these assets were shared, I could see Project Tamriel getting a possible jumpstart on the other side of the Velothi mountains as well as Winterhold-hold even. This would limit the amount they'd need to make come time to work on those exteriors. That would definitely help the whole area feel more unified between projects and feel like a more cohesive Tamriel.
Just curious how possible that would be or how much discussion there had been on that subject.
Just curious how possible that would be or how much discussion there had been on that subject.
I didn't even know this thread existed til now, go me. Anyway, I've been of the same opinion for YEARS that the projects need to start coming together. The sharing of assets would indeed be a great thing, and I also understand that the sharing of lore is important too.
I just don't know if there's a sort of...irc channel or thread on whichever forum where we could discuss consistencies on our created lore and whatnot. There really should be a single place where we can have inter-project cooperation to discuss anything from assets to lore. Lore probably being the most important bit. People are going to want to download a province mod because they want to play ALL of tamriel; not just one of our province mods, so the lore needs to be consistent.
That being said, it's already bad enough that I have to sort through TR's forums for lore and spend so much time and effort just trying to catch up on the lore we've created since I've been gone. I really don't think I have the attention span to try and do this for other province mods as well. Hence, I do think a nice place where we could have these sorts of discussions would be really nice. I'd even sign up on Project Tamriel if that happened.
I just don't know if there's a sort of...irc channel or thread on whichever forum where we could discuss consistencies on our created lore and whatnot. There really should be a single place where we can have inter-project cooperation to discuss anything from assets to lore. Lore probably being the most important bit. People are going to want to download a province mod because they want to play ALL of tamriel; not just one of our province mods, so the lore needs to be consistent.
That being said, it's already bad enough that I have to sort through TR's forums for lore and spend so much time and effort just trying to catch up on the lore we've created since I've been gone. I really don't think I have the attention span to try and do this for other province mods as well. Hence, I do think a nice place where we could have these sorts of discussions would be really nice. I'd even sign up on Project Tamriel if that happened.
Not another memory
...And so my bad karma gets worse
...And so my bad karma gets worse
One of the reasons for the idea of the unified data file is because the projects do share assets. As they have separate data files, however, you end up with two or three versions of the same assets, and for items like misc items and weapons they likely have different stats in the different provinces. When playing the province mods together, that will no doubt prove pretty clunky.
As far as lore is concerned, I agree with everything Not says.
As far as lore is concerned, I agree with everything Not says.
Worsas wrote:Double post:
I'm officially cancelling the data unification idea for the time being. I am not capable to help at it anymore, at this point.
As said I've been hoping, I could do this as the last thing in 2015. But it's failing on resistances inside and outside, so yeah, fuck it, I guess.
I'm sorry to hear this. A unified 'Tamriel' database would be a worthwhile project.
I've mainly written that due to my current stress level. I would love to see a common data myself, I just cannot perform much help at it right now.I'm sorry to hear this. A unified 'Tamriel' database would be a worthwhile project.
It's also not yet clear, what should happen with highres/highpoly opposed to lowres/lowpoly assets. If we go with two separate bsas, as suggested by Anonytroll, I'd like to make use of the extended capabilites of the openmw-engine for the highres-bsa, for example. Might be a good idea to wait on their re-implementation of normal maps, etc. in the osg-version.
Edit: I have bitmap2material installed, which is an incredibly powerful program for the quick creation of good-looking normalmaps and other effect maps based on a single base color texture. I could also see myself doing additional tweaking with substance painter on those textures. <3
Last edited by Worsas on Wed Dec 23, 2015 1:54 pm, edited 1 time in total.
So shall it be done. Discussions on lore and asset usage can now happen at [url=http://www.project-tamriel.com/viewtopic.php?style=4&f=67&t=554]Cross-Project Cooperation Coordination[/url]on the Project Tamriel forums.Not wrote:Hence, I do think a nice place where we could have these sorts of discussions would be really nice. I'd even sign up on Project Tamriel if that happened.
I would appreciate it if someone with knowledge of TR's or SHotN's lore could link the various up-to-date threads and design documents in this topic, so that we can compare notes and get the discussion started. I think the various interpretations of cross-province factions is a priority right now, though any and all points you may have are more than welcome.