Forums | TORN

A plea for a structured and versioned API | API Development | …

Forum Main Main
API Development API
A plea for a structured and versioned API Thread

A plea for a structured and versioned API

pobk
- pobk [3171827]
- Role: Civilian
- Level: 48
- Posts: 51
- Karma: 65
- Last Action: 5 hours
- 9
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Thread created on 14:54:37 - 08/01/24 (8 months ago)

|

Last replied 19:02:17 - 18/01/24 (7 months ago)

Hello,

I'm a bit of a newbie here, but I need to put this out there for my own sanity. Also... Apologies if this has been addressed already somewhere... I'm still digging and finding info on the API

I'm attempting to write an API wrapper around Torn API. The current ones don't support python async IO or response validation in any way that's useful for building structured data flows.

The end goal is to be able to implement a black-box exporter for Prometheus so that I can integrate alertmanager and grafana for visualisation to allow for long-term analysis based on time-series instrumentation and data.

I keep running into headaches around the structure of the data returned and having a stable responses. Either I have to implement optional fields everywhere or I have to implement some kind of state model to work out whether the user I'm querying has migrated - e.g. crime2.0 fields etc.

The Torn Unofficial Docs are... helpful... but also highlight the API inconsistencies.

The problem comes when attempting to instrument the stats and data over a timeseries in a TSDB like Prometheus. Optional fields and other things make the modelling complicated.

So I'm hoping to make a plea for everyone's sanity... as well as all the torn developers.

At a minimum, could the API be versioned? Ideally an OpenAPI specification I can throw a swagger UI at would be amazing, but I realise that would be a hell of a lot of work... so instead, could I please ask nicely if future revisions of the API could be versioned in such a way that we could formalise a schema and model against a specific release of the API? Also, some use flags... for things like Crime2.0 migration status etc?

Frustratedly yours

~ pobk

Last edited by pobk on 15:04:15 - 08/01/24

~ P
Omanpx
- Omanpx [1906686]
- Role: Civilian
- Level: 100
- Posts: 1,934
- Karma: 11,354
- Last Action: 3 hours
- 3
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 15:27:04 - 08/01/24 (8 months ago)
Post link copied to clipboard Copy post link

While i share you sentiments and frustrations, i'd also like to lower your expectations a bit - Ched stated that the API is not an integral part of Torn and we should be happy they are providing it at all. That said, don't expect any major changes, especially by posting in API Dev forum, doubt Ched even visits here. All API related suggestions should go to B&I to even have a chance at being looked at and even then, the API is pretty low on the priority of features the team needs to address. The fact that the new Shoplifting locations took over a week to add to the API endpoint should give you an idea of how unrealistic a versioned and properly documented incarnation of the API is.

Toolbox - a collection of my tools and spreadsheets made for Torn.

Guidebook - a collection of guides i made for Torn.
pobk
- pobk [3171827]
- Role: Civilian
- Level: 48
- Posts: 51
- Karma: 65
- Last Action: 5 hours
- 1
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 15:43:19 - 08/01/24 (8 months ago)
Post link copied to clipboard Copy post link

Omanpx [1906686]
While i share you sentiments and frustrations, i'd also like to lower your expectations a bit - Ched stated that the API is not an integral part of Torn and we should be happy they are providing it at all. That said, don't expect any major changes, especially by posting in API Dev forum, doubt Ched even visits here. All API related suggestions should go to B&I to even have a chance at being looked at and even then, the API is pretty low on the priority of features the team needs to address. The fact that the new Shoplifting locations took over a week to add to the API endpoint should give you an idea of how unrealistic a versioned and properly documented incarnation of the API is.

Don't get me wrong, I have zero expectations.

From what I've read here, in documentation, and in other codebases/api based projects... there's a lot of frustration around the API...

That said, however, the API is a part of Torn. It's being provided. Whether it's integral or not has nothing to do with it - an entire economy has been built up around it, so it is, in fact, a defacto component.

Although your comments do help shed some light on the bigger picture.

~ P
TDup
- TDup [2660552]
- Role: Civilian
- Level: 100
- Posts: 2,485
- Karma: 3,011
- Last Action: 2 minutes
- 2
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 21:25:34 - 09/01/24 (8 months ago)
Post link copied to clipboard Copy post link

I can only agree with what Omanpx said above.

The first step to improve the API, I think, would be to indeed support versioning + support several versions at the same time.
Deprecate old API types, give time to developer to update. Remove deprecated endpoint a couple of months later.

This requires time, good developments practices, and to focus on this instead of other part of the project.

In Torn, API can change at any point, without any kind of warning. A bunch of us got caught by unexpected changes, that broke some part of our tools. (who doesn't like to wake up with messages from 20+ user telling you your script doesn't work anymore, when you are preparing your kids to go to school + work :D :D)

I think what best sum up the situation is this quote from Ched, coming from a very interesting thread.

Use the API at your own risk, expect very minor things to change unexpectedly every now and then.
Never fully trust the structure provided in the logs API, this is provided as a courtesy and will occasionally change without warning.

Have you tried my scripts ?

Battle Stats Predictor

Leslie Wheels Profitability
JKyte
- JKyte [2417425]
- Role: Civilian
- Level: 100
- Posts: 713
- Karma: 901
- Last Action: 8 minutes
- 1
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 19:47:36 - 13/01/24 (7 months ago)
Post link copied to clipboard Copy post link

In addition to what others have said, I would highlight that the torn API provides an excellent opportunity to write thorough integration tests. A few simple functions that will provide clear output of when things change. That by itself is a useful skill.

None of that is said to minimize your frustration with the random changes, those are indeed frustrating but it's part of the programmer life.
pobk
- pobk [3171827]
- Role: Civilian
- Level: 48
- Posts: 51
- Karma: 65
- Last Action: 5 hours
- 0
  
  1
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 21:33:21 - 17/01/24 (7 months ago)
Post link copied to clipboard Copy post link

Ok. I'm a newbie to Torn... but I can't help but some of these replies and want to head-desk.

Why do you just accept things like this because "that's the way it's always been". That may have been true at one point... but frankly, with the amount of user-scripts, sites and other things I've looked into over the last week... One thing is very clear.

The API is FAR more important to the Torn community and that we shouldn't just be just accepting of the status quo.

Seriously, just take a look at how many Excel spreadsheets, gsheets, JS based sites, user plugins, bots, reporting websites and other things that have sprung up around it.

A simple api.torn.com/v12345/:section/ would be a start. How complicated that might be, I don't know... Only the devs might be able to contribute.

~ P
Omanpx
- Omanpx [1906686]
- Role: Civilian
- Level: 100
- Posts: 1,934
- Karma: 11,354
- Last Action: 3 hours
- 2
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 08:27:41 - 18/01/24 (7 months ago)
Post link copied to clipboard Copy post link

pobk [3171827]
Ok. I'm a newbie to Torn... but I can't help but some of these replies and want to head-desk.

Why do you just accept things like this because "that's the way it's always been". That may have been true at one point... but frankly, with the amount of user-scripts, sites and other things I've looked into over the last week... One thing is very clear.

The API is FAR more important to the Torn community and that we shouldn't just be just accepting of the status quo.

Seriously, just take a look at how many Excel spreadsheets, gsheets, JS based sites, user plugins, bots, reporting websites and other things that have sprung up around it.

A simple api.torn.com/v12345/:section/ would be a start. How complicated that might be, I don't know... Only the devs might be able to contribute.

Why do you assume the dev community is just taking it as it is? People have been playing this for nearly 20 years, do you seriously think this discussion has not been brought up earlier? You say you are a noob to Torn yourself - try reading through some of Cheds replies, maybe that will give you a better answer why player responses are like they are. Ched is as stubborn as it gets and he could give two shits what the dev community wants.

As for how hard it is to implement - you probably do not realize just how janky the code that holds Torn together is. The whole system is going through a major rebuild right now, after nearly 20 years. All we can do is hope that the API is not on the bottom of the to-do list for developers to update as well.

Toolbox - a collection of my tools and spreadsheets made for Torn.

Guidebook - a collection of guides i made for Torn.
pobk
- pobk [3171827]
- Role: Civilian
- Level: 48
- Posts: 51
- Karma: 65
- Last Action: 5 hours
- 0
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 09:57:43 - 18/01/24 (7 months ago)
Post link copied to clipboard Copy post link

pobk [3171827]
Ok. I'm a newbie to Torn... but I can't help but some of these replies and want to head-desk.

Why do you just accept things like this because "that's the way it's always been". That may have been true at one point... but frankly, with the amount of user-scripts, sites and other things I've looked into over the last week... One thing is very clear.

The API is FAR more important to the Torn community and that we shouldn't just be just accepting of the status quo.

Seriously, just take a look at how many Excel spreadsheets, gsheets, JS based sites, user plugins, bots, reporting websites and other things that have sprung up around it.

A simple api.torn.com/v12345/:section/ would be a start. How complicated that might be, I don't know... Only the devs might be able to contribute.
Omanpx [1906686]
Why do you assume the dev community is just taking it as it is? People have been playing this for nearly 20 years, do you seriously think this discussion has not been brought up earlier? You say you are a noob to Torn yourself - try reading through some of Cheds replies, maybe that will give you a better answer why player responses are like they are. Ched is as stubborn as it gets and he could give two shits what the dev community wants.

As for how hard it is to implement - you probably do not realize just how janky the code that holds Torn together is. The whole system is going through a major rebuild right now, after nearly 20 years. All we can do is hope that the API is not on the bottom of the to-do list for developers to update as well.

I don't assume anything, but at the same time I've never been able to accept or stomach the "it is what it is" response... To anything. It's a dumb response and something of a problematic stance to take because it implies benign acceptance.

I have been reading a lot of forum posts including Ched's and I agree with your assessment regarding the dev community... that said... It's not a small community anymore. There are people out there doing some wacky things with what we have available...

...and I totally get it. If PHP is still the core codebase, then I can be fairly certain it's a mess - unless you review every line of code the language breeds messy codebases. (The only codebase I have encountered that was written in PHP that wasn't a mess was a very recent app, written by a bunch of hardened PHP devs that were so anally retentive about their code formatting, standards, code policies and other things, you could reconstruct the majority of their code from the meeting notes.)

So I'm going to continue to moan and discuss this and raise B&Is against what I find and issues I experience... I'm also a stubborn, slightly spectrum, slightly adhd... and VERY annoying.

Last edited by pobk on 09:58:21 - 18/01/24

~ P
Kwack
- Kwack [2190604]
- Role: Civilian
- Level: 15
- Posts: 1,865
- Karma: 2,838
- Last Action: 6 hours
- 2
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 11:41:09 - 18/01/24 (7 months ago)
Post link copied to clipboard Copy post link

The API has never traditionally been profitable for Torn whatsoever, it was always provided as-is and was barely used outside of things like DocTorn and TornStats. Tools that made our lives easier, but hopefully not tools we were dependent on.

That's drastically changed in the last few years as there's been huge shifts in the tools people use - TornTools replaced DocTorn, YATA as an alternative to TornStats, the introduction of PDA, etc. And more recently, new scripts/tools popping up weekly.

To a new player now it may seem like Torn is dependent on these tools, and you may even be right in portions of the playerbase, but it's still not viewed as profitable by management.

And they're probably right - a major API rework would cost money and time, and probably wouldn't benefit the playerbase as a whole. Yes it would make our (dev's) lives easier, but we're a minority here.
pobk
- pobk [3171827]
- Role: Civilian
- Level: 48
- Posts: 51
- Karma: 65
- Last Action: 5 hours
- 0
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 14:31:37 - 18/01/24 (7 months ago)
Post link copied to clipboard Copy post link

To a new player now it may seem like Torn is dependent on these tools, and you may even be right in portions of the playerbase, but it's still not viewed as profitable by management.

If Torn were API-first, this would absolutely be true, but to an experienced eye, you can see that the API is as most have described. A second-thought. Although after everything I've read, I don't think it has ever been a priority... and historically, that may have been OK.

My contention is that this is now no longer the case.

From my (limited - read; take with a pinch of salt) perspective, you've got, for example, factions using the API data to record activity and rank member activity, document participation, manage member pay outs... Players using API Data to find leveling targets... Check stock levels in travel destinations... Build training plans for leveling up... and those are just some of the things that I've seen that are cool.

And they're probably right - a major API rework would cost money and time, and probably wouldn't benefit the playerbase as a whole. Yes it would make our (dev's) lives easier, but we're a minority here.

I'm not advocating for a whole re-write... but some stabilised versioning would be grand!

~ P
Omanpx
- Omanpx [1906686]
- Role: Civilian
- Level: 100
- Posts: 1,934
- Karma: 11,354
- Last Action: 3 hours
- 1
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 15:41:30 - 18/01/24 (7 months ago)
Post link copied to clipboard Copy post link

pobk [3171827]

To a new player now it may seem like Torn is dependent on these tools, and you may even be right in portions of the playerbase, but it's still not viewed as profitable by management.

If Torn were API-first, this would absolutely be true, but to an experienced eye, you can see that the API is as most have described. A second-thought. Although after everything I've read, I don't think it has ever been a priority... and historically, that may have been OK.

My contention is that this is now no longer the case.

From my (limited - read; take with a pinch of salt) perspective, you've got, for example, factions using the API data to record activity and rank member activity, document participation, manage member pay outs... Players using API Data to find leveling targets... Check stock levels in travel destinations... Build training plans for leveling up... and those are just some of the things that I've seen that are cool.

And they're probably right - a major API rework would cost money and time, and probably wouldn't benefit the playerbase as a whole. Yes it would make our (dev's) lives easier, but we're a minority here.

I'm not advocating for a whole re-write... but some stabilised versioning would be grand!

factions using the API data to record activity and rank member activity, document participation, manage member pay outs

All this can be done manually and is not strictly necessary - not a lot of factions use the API to manage payouts, in most cases, they just go with the war report from Torn. Your experience in this is indeed very limited. I'm speaking as someone who has developed multiple tools and utilities for all this and the demand is A LOT lower than you make it out to be.

Players using API Data to find leveling targets

This is just straight up incorrect - perhaps 99% of players looking for leveling targets use publicly shared lists of targets that have been compiled without using the API but with stat spies of inactive players.

Check stock levels in travel destinations

Another strongly exaggerated statement - most people just fly blindly to their destination (usually the one that lines up with their nerve / energy / drug cooldowns) and hope to get stock there. While YATA and Torn Tools provide foreign stock data, barely anyone checks that prior to flying, as flying is torture and people just go auto pilot with it.

Build training plans for leveling up

Another thing that is nice to have, but absolutely not a must and not that many people utilise it (speaking as someone who has guides on this shit...). Using Hank's / Baldr's ratio is enough for most people, and that does not use the API.

While the use cases you mentioned are perhaps the least impactful to the game, there are entire business models built around the API, such as merc contracts, Flight Club (huge plushy / flower trading platform), and basically any trader price sheet. Not to mention us developers who rely on API as their main source of income. That said, this was never part of the original game design by Ched and knowing him, it wouldn't cause him much pain if the whole thing just went away. He might wake up when a large part of the player base left the game after that, but as it is right now, judging by his comments and tone, he does not see the API as a vital part of the game. Especially not now with all the chaos of Chat 2.0 and many 2.0 systems being released in the near future.

So i guess what i'm trying to say is, you should first get a slightly more accurate picture of just how much the API is integral to the game before feeling attacked. Also, you literally couldn't have picked a worse time to "moan and discuss" about this topic. The whole Torn 2.0 evolution is in full swing and none of the developers will as much as lift a finger to fix the API for quite some time, not at least the new systems are stable and functioning as intended. Also, full disclaimer - making stuff using the API is my main source of income and the main reason i play this game by far, so don't think i'm against the idea or anything, i'm just trying to be realistic about it.

Toolbox - a collection of my tools and spreadsheets made for Torn.

Guidebook - a collection of guides i made for Torn.
Wolverine
- Wolverine [1971836]
- Role: Civilian
- Level: 100
- Posts: 4,967
- Karma: 5,600
- Last Action: 2 hours
- 1
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 17:08:03 - 18/01/24 (7 months ago)
Post link copied to clipboard Copy post link

pobk [3171827]
Ok. I'm a newbie to Torn... but I can't help but some of these replies and want to head-desk.

Why do you just accept things like this because "that's the way it's always been". That may have been true at one point... but frankly, with the amount of user-scripts, sites and other things I've looked into over the last week... One thing is very clear.

The API is FAR more important to the Torn community and that we shouldn't just be just accepting of the status quo.

Seriously, just take a look at how many Excel spreadsheets, gsheets, JS based sites, user plugins, bots, reporting websites and other things that have sprung up around it.

A simple api.torn.com/v12345/:section/ would be a start. How complicated that might be, I don't know... Only the devs might be able to contribute.
Omanpx [1906686]
Why do you assume the dev community is just taking it as it is? People have been playing this for nearly 20 years, do you seriously think this discussion has not been brought up earlier? You say you are a noob to Torn yourself - try reading through some of Cheds replies, maybe that will give you a better answer why player responses are like they are. Ched is as stubborn as it gets and he could give two shits what the dev community wants.

As for how hard it is to implement - you probably do not realize just how janky the code that holds Torn together is. The whole system is going through a major rebuild right now, after nearly 20 years. All we can do is hope that the API is not on the bottom of the to-do list for developers to update as well.

when I have an issue with torn api, I send you a message and you usually find a solution to my problem :P

@Pobk, create a thread in the API forum and someone usually helps you with a workaround if the developers failed to fix it.

I aften opened bug reports but almost always they close it without solving or they say that they fixed it but it's not true.

Anyway I'm grateful with Chedburn for giving us some api to play with, that's the main reason for which I still play this game.
pobk
- pobk [3171827]
- Role: Civilian
- Level: 48
- Posts: 51
- Karma: 65
- Last Action: 5 hours
- 0
  
  0
- Reason:
  
  Are you sure you want to report this post to staff?
  Cancel
Posted on 19:02:17 - 18/01/24 (7 months ago)
Post link copied to clipboard Copy post link

before feeling attacked.
I'm not feeling attacked at all - befuddled is probably more accurate as to my current position on responses and replies. That said your comments and this whole (continued) discussion is insightful and helpful... and the examples I've posted are the ones I've had the majority of my interaction with and/or the tangents by brain has thought of and the subsequent googling and discovery process.

I would however point out that everything you've said just supports my contention that the API should no-longer be a second-thought and should be more of a first-class citizen in Torn.

however, I would argue that if Torn2.0 is in full swing, then this is absolutely the time to bring it up. Totally the right time to propose [, moan] and discuss API changes that we (the community) want. Don't ask, don't get.

I'm not butthurt. I'm asking questions and pointing the snudges on the floor because I don't know any better.

~ p

~ P