[ home ] [ pony / rp / canterlot ] [ arch ]

/canterlot/ - Canterlot

Site related staff board
Password (For file deletion.)

[Return][Go to bottom]


File: 1499990008544.png (137.75 KB, 900x900, 1:1, 1359885447299.png) ImgOps Google

Does Ponyville have an API?


File: 1500161890748.jpg (72.43 KB, 524x355, 524:355, v104-02_i6-524x355.jpg) ImgOps Exif Google

Does Ponyville have an apiary?


Probably. Keeping bees seems like something Rose would do.


File: 1500202417270.png (168.94 KB, 507x454, 507:454, shrug 3.png) ImgOps Google



Starshine a qt

That wasn't me ^_^

but I do keep bees sorta. Not my own any more, but other people's


File: 1500752726742.jpg (34.83 KB, 519x454, 519:454, Tense.jpg) ImgOps Exif Google

Can I request that a developer add a boardlist to the API, at the URL:
to match that of Ponychan at this URL:

It's for science, and convenient posting!


File: 1500789252387.png (181.24 KB, 900x1099, 900:1099, Apple_Bloom_with_an_Apple.png) ImgOps Google

>>1143  I have to disagree.


Why so?


File: 1500805449572.png (404.96 KB, 1080x1920, 9:16, device-2017-07-23-032302.png) ImgOps Google

Actually, scratch the boardlist thing. The more important question is, can someone help me get around the anti-spam measures?

The reason I ask is because, well…



testing antibot request things


File: 1500824009006.jpg (18.01 KB, 346x300, 173:150, confus.jpg) ImgOps Exif Google

>The reason I ask is because, well…
Nani? I don't understand.


File: 1500828116657.gif (56.46 KB, 391x319, 391:319, 210.gif) ImgOps Google

>>1151  it's a decent-looking mobile version of the site, as opposed to the current dogshit version.


who's that faggot on the button down there posting that pink girl?


File: 1500847505532.jpg (230.36 KB, 1280x1280, 1:1, DJ-,,,-Mornin'.jpg) ImgOps Exif Google

Not that I'm aware of. I'll check with the other developers.

What exactly is triggering the anti-spam measures?


File: 1500883973547.png (108.56 KB, 282x287, 282:287, Screenshot from 2015-06-14….png) ImgOps Google

Ly and I are developing an Android app (forked from Clover, probably the most popular 4chan app) to browse and post on Ponychan and Ponyville.


Seems to be a hash being checked when posting.


File: 1500998321838.png (157.54 KB, 435x360, 29:24, you are wonderful.png) ImgOps Google

>Ly and I are developing an Android app … to browse and post on Ponychan and Ponyville.

WOW, that is -so cool-!!

i like the idea a lot c: is there anything we can do to help out with that? Maybe i can talk to Mikie?


File: 1500999351422.png (165.78 KB, 381x288, 127:96, 1439853132678.png) ImgOps Google

Currently, the number one priority for the Ponyville side is getting posting to work. Ponyville already has the vichan API in place, which serves up raw HTML rather than the text entered directly in the field, which made parsing and loading the data from the site much easier – viewing Ponyville works for the most part, though there are a few errors to work out.

Posting, though, is halted by the site's anti-bot measures, which seem to be a bit more strict than those currently on place on Ponychan. The most helpful thing right now would be if someone with knowledge of the Ponyville backend could help me figure out how to get around those.


File: 1501006116441.png (257.1 KB, 460x616, 115:154, 7878768967678646565.png) ImgOps Google

so far these seem to working great as it seems

i tested flutter on my end and it's pretty convenient

you're gonna like it

>which seem to be a bit more strict than those currently on place on Ponychan
that's kinda odd to me. i would have expected macil to have it set up beyond strict

in anycase will the pville one be called flutter too?


File: 1501006477456.gif (1.84 MB, 960x540, 16:9, 1496474704577.gif) ImgOps Google

I think most of what Macil did was apply filters to capture the CP bot's generation algorithms. There's also a cookie with a userID, which Flutter currently gets around by generating its own userID.

But on Ponyville, there is no such cookie that I'm aware of, and as far as I can tell the bot detection works by checking which fields of a form are submitted when a post is made, and filling up several hidden fields itself. If the hidden fields match what the server is expecting, the post goes through. If not, "Your request looks automated; post discarded."

Currently I'm trying to get around that by sending a blank post to the server first, to get a response that includes some of those hidden fields. But I'm hoping there's a simpler way, because parsing all those hidden fields is going to be a pain to automate – which makes sense, given that the goal is to keep bots out.


>in anycase will the pville one be called flutter too?
Both sites will be available on the same app, for convenience both from a user and developer standpoint. Most of the code to ensure they don't conflict is already in place, and later down the line I'll probably have a dialog on the first startup asking if you want Ponychan boards, Ponyville boards, or both.

One thing to note is that when the first build supporting Ponyville is released, people who have the Ponychan version will have to clear their existing app data – this is because modifications will have been made to the database to accommodate the Ponyville boards.


File: 1501007022939.png (338.16 KB, 572x616, 13:14, Screenshot from 2016-06-03….png) ImgOps Google

ah makes sense. im not too familiar with technical side of things

okay, fair enough. and hey, maybe this will encourage more folks to see that pchan really isn't scary anymore since i am trying my damn best to revive our community again, or at the very least keep it sustainable…


File: 1501007280146.png (229.76 KB, 406x446, 203:223, 1445838967110.png) ImgOps Google

And on the flipside, maybe it'll help people on Ponychan realize that Ponyville isn't some sort of "Our Town" cult.

Still, if people don't care for Ponychan boards, or Ponyville boards, they don't have to have them in their boardlist. Freedom of choice and all. I'm not here to force people to go to either site, just to make posting on the go more convenient for both.


File: 1501007431719.png (268.83 KB, 533x614, 533:614, 6778868.png) ImgOps Google

fair enough then


File: 1501007590817.png (82.17 KB, 300x300, 1:1, 9p08bgfqir9c.png) ImgOps Google

It will be nice if this improves relations between the communities, but I'm not going to force it.


File: 1501007718786.png (157.36 KB, 504x423, 56:47, 86798989879.png) ImgOps Google

there's nothing we could do to do such a thing anyway

but it's something i would like to see

we all come from the same roots after all


POST /post.php HTTP/1.1
User-Agent: Flutter/v2.3.0-d25ddaf
Referer: http://ponyville.us/test/res/21.html
Content-Type: multipart/form-data; boundary=2b10df22-c894-4792-9566-5af794ca3daa
Content-Length: 2882
Host: ponyville.us
Connection: Keep-Alive
Accept-Encoding: gzip

Content-Disposition: form-data; name="name"
Content-Length: 9

Content-Disposition: form-data; name="email"
Content-Length: 4

Content-Disposition: form-data; name="subject"
Content-Length: 0

Content-Disposition: form-data; name="body"
Content-Length: 11

Please work
Content-Disposition: form-data; name="password"
Content-Length: 15

Content-Disposition: form-data; name="on3w4d8bzyaves9x1"
Content-Length: 5

Content-Disposition: form-data; name="hash"
Content-Length: 40

Content-Disposition: form-data; name="0i…xuow8eylfag…pnhs59kqb71jm6…v…c42r"
Content-Length: 101

%b\Rf=NPHWpd`^ …CY.vFlmsq7[ur… a1…ny>Xk*:…Sg……z)…o5,!4K&';9D?Z~i]@3|cT+G…8(/{t
Content-Disposition: form-data; name="q"
Content-Length: 44

Content-Disposition: form-data; name="search"
Content-Length: 55

Content-Disposition: form-data; name="firstname"
Content-Length: 49

Content-Disposition: form-data; name="board"
Content-Length: 4

Content-Disposition: form-data; name="d9siq…xanur…jeg5…3f476vzhoc0wkb1"
Content-Length: 5

Content-Disposition: form-data; name="embed"
Content-Length: 0

Content-Disposition: form-data; name="gp61s72av8mednt43q5yzc…rlxiu0"
Content-Length: 48

…tv(<ZKDyo+/fnV[5%u}*9E…, zU
Content-Disposition: form-data; name="thread"
Content-Length: 2

Content-Disposition: form-data; name="username"
Content-Length: 77

Content-Disposition: form-data; name="4…mboktjf7v3hq…enuzdps…"
Content-Length: 27

Content-Disposition: form-data; name="json_response"
Content-Length: 1

Content-Disposition: form-data; name="post"
Content-Length: 4

HTTP/1.1 200 OK
Date: Wed, 26 Jul 2017 16:26:18 GMT
Content-Type: text/json; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Set-Cookie: __cfduid=d26005dffce212e277c6cbcc617abb98b1501086378; expires=Thu, 26-Jul-18 16:26:18 GMT; path=/; domain=.ponyville.us; HttpOnly
X-Powered-By: PHP/7.0.10
Server: cloudflare-nginx
CF-RAY: 3848b848e0c039c4-PHX

{"error":"Your request looks automated; Post discarded."}


POST /post.php HTTP/1.1
X-Requested-With:        XMLHttpRequest
User-Agent:        Mozilla/5.0 (X11; Linux x86_64; rv:45.9) Gecko/20100101 Goanna/3.2 Firefox/45.9 PaleMoon/27.3.0
Referer:        https://ponyville.us/test/res/21.html
Pragma:        no-cache
Host:        ponyville.us
DNT:        1
Content-Type:        multipart/form-data; boundary=—————————729653893970255723729742731
Content-Length:        2906
Connection:        keep-alive
Cache-Control:        no-cache
Accept-Language:        en-US,en;q=0.5
Accept-Encoding:        gzip, deflate
Accept:        application/json, text/javascript, */*; q=0.01

__cfduid:        d87f7ecf593c5156cea33f9585a6e7e111488111754

—————————–729653893970255723729742731 Content-Disposition: form-data; name="q" O(LT?8vu6W\/:1e$[;0Jx| —————————–729653893970255723729742731 Content-Disposition: form-data; name="thread" 21 —————————–729653893970255723729742731 Content-Disposition: form-data; name="d9siq⚞xanur☠jeg5♭3f476vzhoc0wkb1" 60320 —————————–729653893970255723729742731 Content-Disposition: form-data; name="board" test —————————–729653893970255723729742731 Content-Disposition: form-data; name="gp61s72av8mednt43q5yzc⛣rlxiu0" ☕tv(<ZKDyo+/fnV[5%u}*9E⛫, zU —————————–729653893970255723729742731 Content-Disposition: form-data; name="search" {:SbI]W^}n,.huz=cw[CHaU_5f93vYF(P)gXsxKd>J%@N2A♺;y —————————–729653893970255723729742731 Content-Disposition: form-data; name="name" Starshine#g!-Hi,'= —————————–729653893970255723729742731 Content-Disposition: form-data; name="0i♠xuow8eylfag♎pnhs59kqb71jm6♟v⚟c42r" %b\Rf=NPHWpd`^ ☎CY.vFlmsq7[ur⛲ a1☊ny>Xk*:☬Sg♙⛼z)⛊o5,!4K&';9D?Z~i]@3|cT+G⚇8(/{t —————————–729653893970255723729742731 Content-Disposition: form-data; name="username" in:!H~]xhqwT`84v⚢=}-/G&2♪B⚙p(YA)d;+ubLg*. —————————–729653893970255723729742731 Content-Disposition: form-data; name="email" —————————–729653893970255723729742731 Content-Disposition: form-data; name="firstname" J_]jI*w:38euY!R,HS? —————————–729653893970255723729742731 Content-Disposition: form-data; name="4☴mboktjf7v3hq☱enuzdps⛐" Ys2)*>$_[,ln+GCz<JHp] —————————–729653893970255723729742731 Content-Disposition: form-data; name="on3w4d8bzyaves9x1" 79686 —————————–729653893970255723729742731 Content-Disposition: form-data; name="subject" —————————–729653893970255723729742731 Content-Disposition: form-data; name="body" Test —————————–729653893970255723729742731 Content-Disposition: form-data; name="file"; filename="" Content-Type: application/octet-stream —————————–729653893970255723729742731 Content-Disposition: form-data; name="embed" —————————–729653893970255723729742731 Content-Disposition: form-data; name="password" IqcwSUX# —————————–729653893970255723729742731 Content-Disposition: form-data; name="hash" e7d02ff040d9364be560ec148d02492cf728e3f3 —————————–729653893970255723729742731 Content-Disposition: form-data; name="json_response" 1 —————————–729653893970255723729742731 Content-Disposition: form-data; name="post" Post —————————–729653893970255723729742731–

HTTP/1.1 200 OK
x-powered-by:        PHP/7.0.10
X-Firefox-Spdy:        h2-16
x-associated-content:        "/test/res/21.html#5512"
Server:        cloudflare-nginx
Date:        Wed, 26 Jul 2017 16:32:26 GMT
Content-Type:        text/json; charset=utf-8
CF-RAY:        3848c140c91339ac-PHX



POST request from the app and response from the server: >>1168

POST request from Pale Moon and response from the server: >>1169

What's going wrong? Should I be sending the __cfduid too?


File: 1501126602682.png (157.54 KB, 435x360, 29:24, you are a wonderful pony.png) ImgOps Google

i would like to see us grow closer too! i like the current trajectory of Ponychan, and while Ponyville loves its independence, i think ponies should be friends, not rivals, if at all possible.

we must plant the seed of friendship, while the soil is fertile.


Since you asked on /test/ in reference to this, your issue appears to be one of encoding. the blocks like:

Content-Disposition: form-data; name="0i…xuow8eylfag…pnhs59kqb71jm6…v…c42r"

look like incorrectly encoded (where the ellipses are). Compare this too what your browser sent; which is correctly encoded.

Content-Disposition: form-data; name="0i♠xuow8eylfag♎pnhs59kqb71jm6♟v⚟c42r"

Your going to have to convince java/android to send those strings raw.


Ooh, I didn't even notice that. Nice catch.


I just tested it, and it seems that in Java itself those characters are being encoded correctly, or at least they display correctly in my debug output; they're just showing up as ellipses in the request because I'm logging the requests from Wireshark. Any other ideas?


Update: It's definitely not the encoding.



In Wireshark, the non-ASCII characters in that same field are
\342\230\273, \342\233\260, \342\231\277
respectively, and everything else in the string is the same.


I don't remember exactly how it works, but in Firefox, you can watch a HTTP transaction with everything sent in it, and then resend it. If you do that, you can verify that Ponyville doesn't require the browser to read some kind of randomly generated anti-bot string.


File: 1507431714569.png (83.53 KB, 540x540, 1:1, 1082280__safe_artist-colon….png) ImgOps Google

I hadn't forgotten about this. Having looked at it a couple of times now though and I'm not sure I can say much more than you already know. Some of the problem is that PVille mangles what you paste into it itself. So even your pale moon request is invalid if I just copy what's pasted in >>1169

Using valid tokens from a request I've captured myself though, there doesn't seem to be any problems in the content or header of your flutter app though. It really just does seem to be in either how you're scraping or sending the tokens. It's worth noting each thread has it's own set of tokens. You can't use tokens for one thread to post to another.

I doubt I could do any more without knowing how your app or the website functions more intimately or being smarter. If you're still having difficulty, I recommend using a HTTP proxy yourself like Burp Suite or OWASP ZAP. You should be able to configure your browser or device to point to these proxies running on your computer so you can test your app on your phone and still capture and edit the traffic. Wireshark is far too low level and doesn't have to tools to easily edit and resend http traffic.


File: 1508959501654.png (269.33 KB, 670x670, 1:1, ThinkingRarity.png) ImgOps Google

Development has kind of fallen into limbo currently, in part because of school. I do remember, though, that what I was trying involved submitting a blank POST request to the thread page (which will return a complete HTML document of an error page, which conveniently includes the hidden fields of that thread), and then populating those fields and submitting another POST request.

I never did push the latest revision to GitHub, but that's in part because much of the work done on that revision is kind of...hacked together. Particularly the blank POST request thing, which honestly seems like a smoking gun to me, but it was the only way I managed to actually extract anything of use from the server response. I can push it if you'd want to have a look at it, and promise not to judge.


maybe. You should be able scrape the tokens from any normal page GET request. But if your POST method works then who cares really?

I don't actually have an android tool chain, but I would be happy to read or test the relevant classes if you want.

[Return] [Go to top]
[ home ] [ pony / rp / canterlot ] [ arch ]