A little project of mine

So I've attempted this in the past, and I'm going to attempt it again.

You know those graphs that gaia provides in the marketplace? I'd like to interpret them programmatically. I can take lessons from captcha breaking software to learn to read the text and then I'll have to read the lines separately which should be easy. More worried about getting the x and y axis counts rather than reading the graph itself right now. I shall post sources I create.

Update 1:

Example image, I just want trk on the units sold right now, not the cost graph.



Output similar to what Im looking for: (UPDATED DUE TO UPDATE 2, time per tick is not included anymore so its been removed.)

{

	0: 2,

	1: 5,

	2: 4,

	3: 7,

	etc,

	etc,

	etc

}

A single array. Each numbered object is a tick. Each tick is a vertical line from the x axis, each representing a specific amount of time on the graph. This amount of time is to be calculated and store in the array as "timePerTick" formatted as a double in days.

Also, ticks are numbered from most recent to oldest. Aka right to left.

Update 2:

Actually, pretty sure time between each tick can only be one of two options based on if you select the 1year graph or the 1 month graph. So I can probably use constants for that. Really in this project, I need to calculate the units sold scale and I'll be golden. Tracking the line will be simple because the of the dark blue border around the edge of the graph.

Yea, on the month view each tick is a day. On year its still a day, but the ticks are a million fold. Nothing inbetween them but more ticks, being impossible to tell the difference between them. Thats alright though, I can just assume each pixel is a day but later I can count how many pixels wide the graph is and thus know how many days in each pixel. Fairly simple.

MattSmith wrote:

So I've attempted this in the past, and I'm going to attempt it again.

You know those graphs that gaia provides in the marketplace? I'd like to interpret them programmatically. I can take lessons from captcha breaking software to learn to read the text and then I'll have to read the lines separately which should be easy. More worried about getting the x and y axis counts rather than reading the graph itself right now. I shall post sources I create.

Update 1:

Example image, I just want trk on the units sold right now, not the cost graph.

Sounds really awesome, can't wait to see what comes of this!

I'll try to contact a few of my Gaia Developer friends and ask them where graph data is populated from, see if there's any straight up GSI code we could use or something to get the raw data haha.

Flareboy323 wrote:

Sounds really awesome, can't wait to see what comes of this!

I'll try to contact a few of my Gaia Developer friends and ask them where graph data is populated from, see if there's any straight up GSI code we could use or something to get the raw data haha.

I doubt it, but worth a shot haha

EDIT:: Also, updated with an output I'd like to achieve.

Mk, thought of a heavy weight method without doing and research on how captcha crackers do their work. If I record 1 of each number 0-9 (10 in total) then keep an image of each number I can pull each number from the graph because they are at a constant location and check each pixel vs the 10 images I have and the highest scoring will be what I use for that position. However, this will take quite a bit of work and its ugly. I don't like it. Most probably a much better way.

Yeah, I doubt I'll find anything, but it's worth a shot to try to find out a bit of how the marketplace works haha.

What I've got to pull the pixel data thus far:

var c = document.createElement('canvas');

var ctx = c.getContext("2d");

var image = new Image();

image.src = "../../../dress-up/avatar/vend/10006349_tot_month.png";

ctx.drawImage(image, 0, 0, image.width, image.height);

var data = ctx.getImageData(0,0,image.width,image.height);

For example, executing it from the marketplace item listings page.

If you do image = document.getElementById ,
you get security errors because gaia doesnt like cross-origin data so you have to pull the url like I have done and set the src for it trk.

ITT: Complete and utter banter and tomfoolery.

Make something for a change then we'll talk, mmkay.

The Unintelligible wrote:

ITT: Complete and utter banter and tomfoolery.

Make something for a change then we'll talk, mmkay.

Lol, can I favorite this like twitter? Its real talk, I don't finish anything, always get distracted haha

That's why we need that Like button back.

I don't know, seeing people programming at all is good enough for me. And hey, it's activity, haha.

Also, really close to getting in contact with an MP dev

Flareboy323 wrote:

I don't know, seeing people programming at all is good enough for me. And hey, it's activity, haha.

Also, really close to getting in contact with an MP dev

Awesome, appreciate the help flare. Any new information is great

Seems like a novel idea.
I second the push for a like button, and I also propose a dislike button.

MattSmith wrote:

What I've got to pull the pixel data thus far:

var c = document.createElement('canvas');

var ctx = c.getContext("2d");

var image = new Image();

image.src = "../../../dress-up/avatar/vend/10006349_tot_month.png";

ctx.drawImage(image, 0, 0, image.width, image.height);

var data = ctx.getImageData(0,0,image.width,image.height);

For example, executing it from the marketplace item listings page.

If you do image = document.getElementById ,
you get security errors because gaia doesnt like cross-origin data so you have to pull the url like I have done and set the src for it trk.

wat

User scripts don't have cross-origin security. Cut out your silly dev bologna part. Or export the security issues to a function within the user script. If needed, I can just do something like __LG__.get("id-here") that would just return the element with that ID. Should give you cross-script access to the element, I'd think.
But I imagine you could do all this on your own. Just make __LG__.botobject.functionThatCircumventsSecurity and call it whenever your external program needs it.

Also if you can read pixel color values and need help interpreting them, I can probably help contribute with that. I love programmatic image manipulation.

GAMEchief wrote:

wat

User scripts don't have cross-origin security. Cut out your silly dev bologna part. Or export the security issues to a function within the user script. If needed, I can just do something like __LG__.get("id-here") that would just return the element with that ID. Should give you cross-script access to the element, I'd think.
But I imagine you could do all this on your own. Just make __LG__.botobject.functionThatCircumventsSecurity and call it whenever your external program needs it.

Also if you can read pixel color values and need help interpreting them, I can probably help contribute with that. I love programmatic image manipulation.

You get security errors, not sure what else to tell you. Ill reenact it later and give you a screenshot, can't right now about to take off.

Also, sounds great, I was planning on implementing this for lgaia, thats why I wrote it in javascript/am writing it in javascript.

EDIT::

Psst, my ultimate goal is to make a marketplace bot that is pro I can use the daily units sold to tell if its worth it to buy something priced like 5% under the item above it and sell it at 2% below that item and make a 1% profit in no time. Very small, but do this a lot, and you will make a good amount of gold. I have taken into consideration the 2% sell fee

Here:



vs




EDIT::

I could be doing it very wrong

A few things:
1) yui-gen# is an automated ID given by the Yahoo API to anything with event listeners. It will not always reference what you are trying to get, since the # is semi-random and based on the order event listeners are executed. Use an identifier that isn't YUI-based.
2) the first parameter of drawImage is supposed to be an element?
3) Is your first code running from your domain, or is it running from inside a userscript? A script added from your domain to the userscript does not count as running from the userscript. Running it entirely within the userscript should remove the security error. And like I said, if you are insistent on running it from your domain, I can put just the secured code into the LGaia object, and then you could interact with the secured content through LGaia.
4) Try using an absolute path for image.src = "/path/to/tot_month.png" instead of using the dot directory navigators.
5) Why does the second script say "undefined" even though data seems to hold a value?

GAMEchief wrote:

A few things:
1) yui-gen# is an automated ID given by the Yahoo API to anything with event listeners. It will not always reference what you are trying to get, since the # is semi-random and based on the order event listeners are executed. Use an identifier that isn't YUI-based.
2) the first parameter of drawImage is supposed to be an element?
3) Is your first code running from your domain, or is it running from inside a userscript? A script added from your domain to the userscript does not count as running from the userscript. Running it entirely within the userscript should remove the security error. And like I said, if you are insistent on running it from your domain, I can put just the secured code into the LGaia object, and then you could interact with the secured content through LGaia.
4) Try using an absolute path for image.src = "/path/to/tot_month.png" instead of using the dot directory navigators.
5) Why does the second script say "undefined" even though data seems to hold a value?

#1, I give no fucks about stability about it right now, I was using it to prove a point.
#2, No clue
#3, LGaia and my code are running in the same space, on the browser.
#4, Thats the exact solution I was using... You just proved me correct
#5, I executed many lines of code in one command, thats just what chrome's dev console does. I didnt print anything.

The 1 year graph has 320 days if it is 1 day per pixel.

MattSmith wrote:

#1, I give no fucks about stability about it right now, I was using it to prove a point.

What point? The element you are getting won't be the same with each pageload. So... I wouldn't be surprised if the code has trouble getting data from the element.

#2, No clue

I don't think it does.

#4, Thats the exact solution I was using... You just proved me correct

Proved what correct? That you had the URL wrong?

#5, I executed many lines of code in one command, thats just what chrome's dev console does. I didnt print anything.

Pretty sure it only does that if a variable you tried using it actually undefined. Ergo nnder the code doesn't result in what you are expecting.

GAMEchief wrote:

What point? The element you are getting won't be the same with each pageload. So... I wouldn't be surprised if the code has trouble getting data from the element.


I don't think it does.


Proved what correct? That you had the URL wrong?


Pretty sure it only does that if a variable you tried using it actually undefined. Ergo nnder the code doesn't result in what you are expecting.

Simple solution to this, since you're bitching at me with 0 lines of code to show from yourself, I have done the testing and you haven't, what I've done is required in order for the method trk. If someone else can prove me wrong, please by all means write the code the prove it.

Didn't you say you don't even know JavaScript that well?

GAMEchief wrote:

Didn't you say you don't even know JavaScript that well?

I do not know javascript that well.

GAMEchief wrote:

Didn't you say you don't even know JavaScript that well?

(The difference is, he knows REAL programming languages, lol)

JavaScript is more or less a "real" programming language. But more specifically, it is a scripting language. Just a scripting language with huge capabilities and pretty much powers the web as we know it.

Either way, your statement only discredits JavaScript. It isn't exactly simple to learn and it's a powerful and capable language. Him knowing C# or the next programming language doesn't mean it's more significant than knowing something like JS.

MattSmith wrote:

I do not know javascript that well.

Then I'm just going to point out when I tell you what's wrong with your code, you probably shouldn't say "nu uh."

Flareboy323 wrote:

(The difference is, he knows REAL programming languages, lol)

JavaScript is a real programming language. I think you are confusing it with a markup language, which is also common in web programming.

Why don't you just try to approximate the values. You don't need to know exactly how many units were sold each day. As long as you have an average idea of how many units were sold during a period of time, that should be enough. Also, what really matters is the demand fluctuation. If demand has been falling for the past week, then make a decision based on that... and so on.

The Unintelligible wrote:

JavaScript is more or less a "real" programming language. But more specifically, it is a scripting language. Just a scripting language with huge capabilities and pretty much powers the web as we know it.

Either way, your statement only discredits JavaScript. It isn't exactly simple to learn and it's a powerful and capable language. Him knowing C# or the next programming language doesn't mean it's more significant than knowing something like JS.

Agreed.

@Riddle - I think approximating would be more difficult than getting the actual numbers. And Indeed the most recent trends are more significant, however I'd like to feed as much data as possible to the programmer.

I guess more data means more informed decisions. I didn't really read through the thread entirely, but if there aren't enough pixels to determine the amount of units sold per day, just divide the amount of days in that particular year (leap years into consideration) by the amount of pixels dividing the graph, and the you can approximate how many units were sold by day... or you could just tell the programmer that every 1.xxx days y amount of units were sold.

MattSmith wrote:

@Riddle - I think approximating would be more difficult than getting the actual numbers. And Indeed the most recent trends are more significant, however I'd like to feed as much data as possible to the programmer.

I think he's just talking about compressing the data, instead of storing every individual day if you don't need every individual day.
{
total: 500,
days: 5,
lastDateLogged: howeveryouwanttostorethissothatyoudon'taccidentallylogthesamedaytwice
}

Average per day: 500 / 5

You can even store average instead of total for larger numbers/days:

{
average: 1000000,
days: 365,
lastDate: etc
}

Adding a new day to this average:
average = (average * days + newDaysValue) / (days + 1);
days++;
lastDate = newEtc;

Saves dem bytes, if you aren't using then anyway.
But I can see how knowing them would be useful (day of the week stats, or progress over time, since an item being worth 10g a year ago and 1m gold today isn't only worth 500k gold on average, but instead the rate of change trend is more important).
Calculating a [curved] line of best fit for a set of data would be insanely useful, especially if you can get the derivative of that line. It'd theoretically create the best predictor of which items are the best to hoard.[/curved]

Taking the average value shorter time periods would be ideal though. You could average value per week (which in my opinion would represent a more accurate data set than individual daily values, due to user-induced inflation and other types of fluctuations). Then use that data to calculate the rate of change.

Didn't read the responses in the thread, but there's a rather simple approach to be had here. Simply use any character recognition library to read the low and high range of your scale (i.e. the y-axis: 0 and 15 in that pic provided), break the graph up in to separate images of width x (the smaller the value of x the greater precision/accuracy you'll have), iterate through the images and scan them for whatever colour they use for the graph (whatever shade of blue that is). You then use that y coordinate, compare it with the scale and voila, you have the value for that period (if you want specific dates do the same, but with the x coordinates).

You really only need to read the number in the top left. The values of each one can be determined by just the number of pixels that are blue divided by the height of the graph times the top left number. Assuming it always starts at 0. Maybe it doesn't.