TF2 Item Metrics#700
Sparked by the Scorch Shot ban discussion, I was interested in learning about the reality of certain pertinent data points. Namely, the probability of the weapon hitting a target twice, as well as the average duration of its afterburn (a histogram visual would be nice).
I think this present an opportunity to discuss more detailed stats/metrics about items in the game. I think it would be interesting to talk about what data/analyses might help us inform decisions we make as a community.
Right now I’m working with two other people in a discord: pm me or reply if you’d like an invite to contribute or learn more or whatever: ideas, connections, skills, knowledge, general support, we’re just exploring these ideas right now. Not sure if I’m allowed to post a link to the discord here.
So far I have created some scripts to download a bunch of invite match log files (and can easily get their logs.tf json). Thanks to requiescas for the match directory with logs. For my own next steps, I will using the log files to infer relevant game state needed to track scorch-shot-related events.
We have some interest in parsing the json from the logstf api as well. Here’s a github link to what I’ve made so far: it’s gross and messy and new, so don’t expect much.
TLDR: Data and analysis for items good. Let’s get more of it. Interested in helping?
Lastly some questions: What kind of metrics are you interested in? Any weapons you would like to see more in-depth stats on?
what information are you looking at finding out exactly? We at Engineer.tf are looking to do something similar with displaying weapon metrics through downloaded log files. My main goal is to show data about weapon usage, accuracies, and damage in comparison to the other weapons, maps, and time periods. For example, being able to see what primaries were run on cp_steel for scout back in season 1 vs season 7. Or what secondaries were were used in the whole of season 6 vs season 7 for pyro.
My lofty dream is a site where one could view many of those details you list, alongside more detailed stats about specific weapon functionality (hence the two I listed at the beginning). As well as being able to set up those parameters in an easy to use fashion. I’m just trying to get the small piece done (stats about SS), but want to know if there’s some way to leverage the work that is done to that end towards that dream of mine. So it sounds like we have a shared/similar goal and some overlapping work, might be good to see what we can piece together between us.
Regarding what you want to get, I currently have a list of log tf IDs for invite matches sorted by season and week for seasons 1 to 7, thanks to requiescas. From there I went ahead and grabbed the 700mb worth of logs for those seasons, labeled by those IDs. Right now I’m seeing if I can do some regex on the logs to parse them, but it’s not difficult to query the logstf json api for the higher level stat overview that the site provides. For example, to get the secondaries used by pyros for a season, we can go through each of those ids in that season, query the logstf json api or a cached file, collect that data and do what we can to correct inconsistencies.
I think one challenge is going to be transforming the data into a more usable form, since the json log api isn’t necessarily suited to these kinds of needs. And certainly not the log files themselves.
Another, perhaps more pertinent challenge, is well labeled data sets. The way I parsed the logs given to me by requiescas completely ignores maps, for example, since there is some inconsistency in the labelling (which isn’t meant to be a jab at them, it’s insanely helpful as it is).
@HA-Johnny ngl I am not a coder in any real way, but the way we are going to go about it is downloading them, associating them with our already existing STV demo archive, and then just linking them to start. Once that part is finished, allowing for some form of input box to specify the dataset is the plan. I am personally leaving all the planning to the web devs since I dont really understand how any of this works beyond the basic idea and function.
seems like a really cool project. one thing you might want to be aware of are that in the logs json file if a weapon isn’t fired it will show up as unknown. probably not an issue for the ss, but for some weapons it might be
we ran into this trying to determine medic melee usage for S3 6s, which leaned data towards the ubersaw because it was more likely to be swung
e: idk if this would help you but pazer has an old library for parsing demos, haven’t used it myself though https://github.com/PazerOP/DemoLib2
@ire The one thing I have learned from this over the past couple of days, is that there are like 25 different ways people have independently developed to pull logs for data.
@ire Thanks for that info, and for the library link, I’ll check that out. Will keep that usage info in mind as data is presented and see if there’s maybe some way to backfill that data with predictions or demofile info
@Mothership Maybe get me in touch with the web devs? I suck at Frontend stuff, but maybe I can contribute in some way to backend/science efforts.