This is a good point about what might happen, but it seems worthwhile to address and fix directly. Personally I don't see why adversarial scraping of a publicly published website should require any more ethical consideration/review than using the suggested API would. Ethical concerns should revolve around humans, not the business desires of non-human entities.
Scraping web pages is a last resort for when proper APIs are not available.
Two important take-aways from that:
1) The attackers just pulled the old "Hey, I'm from IT service desk and need your password."
2) Apparently every engineer had god mode in production
I can't imagine what scary monsters are in that codebase.
Of course this used to have the side effect of breaking significant swathes of basic browser UX, especially in areas of accessibility. I assume they’re better now than they once were, but given musk’s historical behavior I assume he won’t consider breaking something like accessibility to be bad or problematic.
I don't see why would he spend more to make website less accessible. Not fixing bugs - yes, just about everyone does so. But breaking it intentionally costs money.
Tesla's aren't THE most accessible (by cost) EV out there, but SpaceX def is.
[1] https://www.wired.com/story/twitter-layoffs-accessibility/
[2] https://www.businessinsider.com/twitter-sent-musk-risks-paid...
SpaceX is... an accessible EV? What?
No, I don't. It obviously is trying to say something about accessibility and SpaceX, but I don't see how the sense of accessbility being discussed in the thread even applies to SpaceX, much less what claim is being made.
Just because it’s the same word does not mean it’s the same meaning.
This is a comment thread discussing accessibility at Twitter, in a post about Twitter discontinuing part of its platform. It’s not a place to circlejerk Musk’s achievements, regardless how much you would want that to be so.
Explain, in exact words, what about SpaceX in any way shape or form would indicate that they know how to handle user a11y and how that would transfer over to Twitter— you know, the thing we’re actually discussing here.
Usage for reading is something like (but not identical to)
1.sh screen_name > 1.json
yy059 < 1.json|grep full_text > 1.txt
less 1.txt
unset connection
export Connection=keep-alive
yy059 < 1.json|grep full_text \
|yy030 \
|grep "https://t.co/.{10}$" \
|uniq \
|yy025 \
|nc -vv h1b 80 \
|sed -n '/location: /s///p' \
|ahref > 1.htm
links -no-connect 1.htm
Alternatively if I do not trust the URLs I might use a script called "www" instead of ahref. It takes URLs on stdin and fetches archive.org URLs wrapped in simple HTML to stdout, using the IA's cdx API.
#!/bin/sh
SCREEN_NAME=$1
COUNT=500
PUBLIC_TOKEN="Bearer AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA"
GT=$(exec curl -A "" -s https://twitter.com/$SCREEN_NAME|sed -n '/gt=/{s/.*gt=//;s/;.*//p;}');
echo "x-guest-token: $GT" >&2;
REST_ID=$(exec curl -A "" -H "authorization: $PUBLIC_TOKEN" -H "content-type: application/json" -H "x-guest-token: $GT" -s "https://twitter.com/i/api/graphql/mCbpQvZAw6zu_4PvuAUVVQ/UserByScreenName?variables=%7B%22screen_name%22%3A%22$SCREEN_NAME%22%2C%22withSafetyModeUserFields%22%3Atrue%2C%22withSuperFollowsUserFields%22%3Atrue%7D"|sed 's/\(rest_id\":\"[0-9]*\)\(.*\)/\1/;s/.*\"//');
echo "rest_id: $REST_ID" >&2;
curl -A "" -H "authorization: $PUBLIC_TOKEN" -H "content-type: application/json" -H "x-guest-token: $GT" -s "https://twitter.com/i/api/graphql/3ywp9kIIW-VQOssauKmLiQ/UserTweets?variables=%7B%22userId%22%3A%22${REST_ID}%22%2C%22count%22%3A$COUNT%2C%22includePromotedContent%22%3Atrue%2C%22withQuickPromoteEligibilityTweetFields%22%3Atrue%2C%22withSuperFollowsUserFields%22%3Atrue%2C%22withDownvotePerspective%22%3Afalse%2C%22withReactionsMetadata%22%3Afalse%2C%22withReactionsPerspective%22%3Afalse%2C%22withSuperFollowsTweetFields%22%3Atrue%2C%22withVoice%22%3Atrue%2C%22withV2Timeline%22%3Atrue%7D&features=%7B%22dont_mention_me_view_api_enabled%22%3Atrue%2C%22interactive_text_enabled%22%3Atrue%2C%22responsive_web_uc_gql_enabled%22%3Afalse%2C%22vibe_tweet_context_enabled%22%3Afalse%2C%22responsive_web_edit_tweet_api_enabled%22%3Afalse%2C%22standardized_nudges_misinfo%22%3Afalse%2C%22responsive_web_enhance_cards_enabled%22%3Afalse%2C%22include_rts%22%3Atrue%7D"
This solution isn't pretty but I can easily keep tabs on Twitter feeds without any need for a Twitter account, a Twitter "API key" or a so-called "modern" browser.
Every person visiting twitter.com is using this same token. The token is neither personal nor private.
What would be the point of changing or blocking it.
Be interesting to see if that stays the same when they're charging for the API but leaving a huge loophole with this token.
Using GraphQL like this can be an effective dark pattern because to anyone using a "modern" browser that "tech" commpanies control it makes it seem like the text of the website cannot be retrieved without Javascript enabled. That's false, but nonetheless it gets people to enable Javascript because the website explicitly asks them to enable it. Then the website, i.e., "tech" company, can perform telemetry, data collection, surveillance, and other shenanigans.
Sometimes this practice might not be a deliberate dark pattern, it might just be developers who are using Javascript gratuitously. For example, HN search provided by Algolia uses GraphQL. HN puts URLs with pre-selected query terms and a public token ("API key") on the HN website. Everyone that uses those URLs uses the same key.
Unlike Twitter, HN istelf does not ask anyone to enable Javascript. The website works fine without it, including the Algolia search, as shown below.
Usage is
1.sh query > 1.json
#!/bin/sh
curl -A "" -d '{"query":"$@","analyticsTags":["web"],"page":0,"hitsPerPage":30,"minWordSizefor1Typo":4,"minWordSizefor2Typos":8,"advancedSyntax":true,"ignorePlurals":false,"clickAnalytics":true,"minProximity":7,"numericFilters":[],"tagFilters":["story",[]],"typoTolerance":"min","queryType":"prefixNone","restrictSearchableAttributes":["title","comment_text","url","story_text","author"],"getRankingInfo":true}' "https://uj5wyc0l7x-3.algolianet.com/1/indexes/Item_production_sort_date/query?x-algolia-agent=Algolia%20for%20JavaScript%20(4.0.2)%3B%20Browser%20(lite)&x-algolia-api-key=8ece23f8eb07cd25d40262a1764599b1&x-algolia-application-id=UJ5WYC0L7X" #!/bin/sh
export Connection=close;
export Content_Type=x-www-form-urlencoded;
export httpMethod=POST;
x=$(echo '{"query":"'$@'","analyticsTags":["web"],"page":0,"hitsPerPage":30,"minWordSizefor1Typo":4,"minWordSizefor2Typos":8,"advancedSyntax":true,"ignorePlurals":false,"clickAnalytics":true,"minProximity":7,"numericFilters":[],"tagFilters":["story",[]],"typoTolerance":"min","queryType":"prefixNone","restrictSearchableAttributes":["title","comment_text","url","story_text","author"],"getRankingInfo":true}');
export Content_Length=${#x};
echo "https://uj5wyc0l7x-3.algolianet.com/1/indexes/Item_production_sort_date/query?x-algolia-agent=Algolia%20for%20JavaScript%20(4.0.2)%3B%20Browser%20(lite)&x-algolia-api-key=8ece23f8eb07cd25d40262a1764599b1&x-algolia-application-id=UJ5WYC0L7X"|(yy025;echo "$x") \
|nc -vv h1b 80