It's really not a useful platform for publicly sharing information anymore. Drives me nuts that government agencies use it for announcements like "Here's an amber alert with a twitter link, but you can't have any of the followup information because that's only for people who are logged in."

You simply see what the author posted and people's reactions.

It also doesn't load 400MB of JavaScript or whatever.

This is a semantic punt from nicer to accessible.

Just call it Twitter.

My God, there's two of us!

(Though … you're being privacy conscious on Chrome? Come to Firefox. Ignore the pesky "it's funded by Google" problems, nothing to see, nothing to see, the water is fiiiine.)

> You might be surprised to learn that normally, this actually works fine

I guess I have a different experience there. A huge number of sites just outright crash. (E.g., the HN search.) JavaScript devs, I've learned, do not handle error cases, and the exceptions tend to just propagated out and ruin the rendering. There seems to be some popular framework out there that even just destroys the whole DOM to emit just the error. (I forget the text, but it's the same text, always. Always centered. Flash of page, then crash.)

I have a custom extension that fakes the cookie storage for those JS pages that just lies & says "yeah, cookies are enabled" and the blackholes the writes. But it fails for anything that needs a real cookie … like Anubis.

I'm empathetic towards where Anubis is coming from though. But the "I passed the challenge" cookie is indistinguishable from a tracker … although probably most people running Anubis are inherently trustworthy by a sort of cultural association so long as Anubis remains non-mainstream. I think I might modify it to have the ability to store cookies for a short time frame (like 1h) in some cases, such as Anubis; that's enough to pass the challenge, but weighed against tracking. I'm usually only blocked by Anubis for something like a blog post, so that should suffice.

Also, after the prescription ends, they're still filling it. I just never pick it up. The autonomous flow has no ability to handle this situation, so now I get a monthly text that my prescription is ready. The actual support line is literally unmanned, and messages given it are piped to /dev/null.

The existing automation is hot garbage. But C-suite would have me believe our Lord & Savior, AI, will fix it all.

mypy's output is, AFAICT, also non-deterministic, and doesn't support a programmatic format that I know of. This makes it next to impossible to write a wrapper script to diff the errors to, for example, show only errors introduced by the change one is making.

Relying on my devs to manually trawl through 80k lines of errors for ones they might be adding in is a lost cause.

Our codebase also uses SQLAlchemy extensively, which does not play well with typecheckers. (There is an extension to aid in this, but it regrettably SIGSEGVs.)

Also this took me forever to understand:

  from typing import Dict

  JsonValue = str | Dict[str, "JsonValue"]

  def foo() -> JsonValue:
      x: Dict[str, str] = {"a": "b"}
      return x

  x: JsonValue = foo()

  example.py:7: error: Incompatible return value type (got "dict[str, str]", expected "str | dict[str, JsonValue]")  [return-value]

Regarding the ~80k errors. Yeah, nothing to do here besides slowly grinding away and adding type annotations and fixes until it's resolved.

For the code example pyright gives some hint towards variance but yes it can be confusing.

https://pyright-play.net/?pyrightVersion=1.1.403&code=GYJw9g...

When done, do typing similarly.

Love this.

Azure I'm looking at you. Many of their services do this, but Blob storage is something else: I've literally gotten information-free responses there. (I.e., 0 B of actual data. I wish I could say 0 B were used to transfer it.)

When you're designing, think about how big a record/object/item is, and return a reasonable number of them in a page. For programmatic consumers who want to walk the dataset, a 640 KiB response is really not that big, and I've seen so many times responses orders of magnitude less, because someone thought "100 items is a good page size, right?" and 100 items was like 4 KiB of data.

> If you have thirty API endpoints, every new version you add introduces thirty new endpoints to maintain. You will rapidly end up with hundreds of APIs that all need testing, debugging, and customer support.

You version the one thing that's changing.

As much as I hate the /v2/... form of versioning, nobody reversions all the /v1/... APIs just because one API needed a /v2. /v2 is ghost town, save for the /v2 APIs.

Not implementing pagination at the outset can be problematic, however. If you later want to paginate data (e.g. if the size of your data grows) then it’s going to be a breaking change to implement that later. Big page sizes but with pagination can be a reasonable balance.

Of course the should be some sort of maximum size, but I have seen APIs that return 1200 lines of text and require me to page them at 100 per request with no option to turn it off.

> Python is a great scripting language, and won't blow your foot off if you try to iterate through an array.

I kind of hate that every time the topic of shell scripting comes up, we get a troop of comments touting this mindless nonsense. Python has footguns, too. Heck, it's absolutely terrible and hacky if you try to do concatenative programming with it. Does that mean it should never be used?

Instead of bashing the language, why not learn bash the language? IME, most of the industry has just absorbed shell programming haphazardly through osmosis, and almost always tries to shove the square pegs of OOP and FP into the round hole that is bash. No wonder people are having a bad time.

In contrast, a data-first design that heavily normalizes data into line-oriented tables and passes information around in pipes results in simple, direct code IME. Stop trying to use arrays and embrace data normalization and text. Also, a lot of pain comes from simply not learning the facilities, e.g. the set builtin obviates most uses of string munging and exec:

    set -- "$@" --file 'filename with spaces.pdf'
    set -- "$@" 'data|blob with "dangerous" characters'
    set -- "$@" "$etc"
    some_command "$@"

C/C++ got us really far, but greenfield projects are moving to safer languages where they can. Expert low level programmers, armed with all of the available linting tools are still making unfortunate mistakes. At some point we should switch to something better.

The senselessness I object to is not the conscientious choice of tooling or discussion of the failings thereof; it's the fact that every single bash article on here sees the same religious refrain, "Python is better than bash. Period." It's like if every article about vim saw a flood of comments claiming that vim is okay for light editing, but for any real programming we should use a real editor like emacs.

If you open vim expecting emacs but with a few different bindings, then it might just explode in you face. If you use bash expecting to be able to program just like Python but with slightly different syntax, then it's not surprising to feel friction.

IME, bash works exceptionally well using a data-oriented, text-first design to program architecture. It's just unfortunate that very little of the industry is even aware of this style of programming.

Agree. Python isn't it though

So in your last example, UTF-8 & UTF-32 are the same type, containing the same infinite set of values, and ­— of course — one can convert between them infallibly.

But you can't encode arbitrary Go strings in WTF-8 (some are not representable), you can't encode arbitrary Python strings in UTF-8 or WTF-8 (n.b. that upthread is wrong about Python being equivalent to Unicode scalars/well-formed UTF-*.) and attempts to do so might error. (E.g., `.encode('utf-8')` in Python on a `str` can raise.)

If you imagine a format that can encode JavaScript strings containing unpaired surrogates, that's WTF-8. (Well-formed WTF-8 is the same type as a JS string, through with a different representation.)

(Though that would have been cute name for the UTF-8/latin1/UTF-8 fail.)

[1]: https://simonsapin.github.io/wtf-8/

(On review, it appears that the thread mentions much earlier uses...)

A quick Google says that solar generates ~20 W/sq ft., so the amount of farmland lost here, by implication, to solar generation, is enough to power the entire United State with solar power alone, twice over.

Obviously, not all 1.2 million acres of land here is lost to solar generation as the government is implying. They don't cite their source, but AFAICT, this is all land that is no longer farmland for any reason at all.

Python's flexible string system has nothing to do with this. Python could easily have had len() return the byte count, even the USV count, or other vastly more meaningful metrics than "5", whose unit is so disastrous I can't put a name to it. It's not bytes, it's not UTF-16 code units, it's not anything meaningful, and that's the problem. In particular, the USV count would have been made easy (O(1) easy!) by Python's flexible string representation.

You're handwaving it away in your writing by calling it a "character in the implementation", but what is a character? It's not a character in any sense a normal human would recognize — like a grapheme cluster — as I think if I asked a human "how many characters is <imagine this is man with skin tone face palming>?", they'd probably say "well, … IDK if it's really a character, but 1, I suppose?" …but "5" or "7"? Where do those even come from? An astute person might like "Oh, perhaps that takes more than one byte, is that it's size in memory?" Nope. Again: "character in the implementation" is a meaningless concept. We've assigned words to a thing to make it sound meaningful, but that is like definitionally begging the question here.

The unit is perfectly meaningful.

It's "characters". (Pedantically, "code points" — https://www.unicode.org/glossary/#code_point — because values that haven't been assigned to characters may be stored. This is good for interop, because it allows you to receive data from a platform that implements a newer version of the Unicode standard, and decide what to do with the parts that your local terminal, font rendering engine, etc. don't recognize.)

Since UTF-32 allows storing every code point in a single code unit, you can also describe it that way, despite the fact that Python doesn't use a full 4 bytes per code point when it doesn't have to.

The only real problem is that "character" doesn't mean what you think it does, and hasn't since 1991.

I don't understand what you mean by "USV count".

> but what is a character?

It's what the Unicode standard says a character is. https://www.unicode.org/glossary/#character , definition 3. Python didn't come up with the concept; Unicode did.

> …but "5" or "7"? Where do those even come from?

From the way that the Unicode standard dictates that this text shall be represented. This is not Python's fault.

> Again: "character in the implementation" is a meaningless concept.

"Character" is completely meaningful, as demonstrated by the fact the Unicode Consortium defines it, and by the fact that huge amounts of software has been written based on that definition, and referring to it in documentation.

Python does not use UTF-32, even notionally. Yes, I know it uses a compact representation in memory when the value is ASCII, etc. That's not what I'm talking about here. |str| != |all UTF32 strings|; `str` and "UTF-32" are different things, as there are values in the former that are absent in the latter, and again, this is why encoding to utf8 or any utf encoding is fallible in Python.

Code points is not a meaningful metric, though I suppose strictly speaking, yes, len() is code points.

> I don't understand what you mean by "USV count".

The number of Unicode scalar values in the string. (If the string were encoded in UTF-32, the length of that array.) It's the basic building block of Unicode. It's only marginally useful, and there's a host of other more meaningful metrics, like memory size, terminal width, graphemes, etc. But it's more meaningful than code points, and if you want to do anything at any higher level of representation, USVs are going to be what you want to build off. Anything else is going to be more fraught with error, needlessly.

> It's what the Unicode standard says a character is.

The Unicode definition of "character" is not a technical definition, it's just there to help humans. Again, if I fed that definition to a human, and asked the same question above, <facepalm…> is 1 "character", according to that definition in Unicode as evaluated by a reasonable person. That's not the definition Python uses, since it returns 5. No reasonable person is looking at the linked definition, and then at the example string, and answering "5".

"How many smallest components of written language that has semantic value does <facepalm emoji …> have?" Nobody is answering "5".

(And if you're going to quibble with my use of definition (1.), the same applies to (2.). (3.) doesn't apply here as Python strings are not Unicode strings (again, |str| != |all Unicode strings|), (4.) is specific to Chinese.)

> "Character" is completely meaningful, as demonstrated by the fact the Unicode Consortium defines it, and by the fact that huge amounts of software has been written based on that definition, and referring to it in documentation.

A lot of people write bad code does not make bad code good. Ambiguous technical documentation is likewise not made good by being ambiguous. Any use of "character" in technical writing would be made more clear by replacing it with one of the actual technical terms defined by Unicode, whether that's "UTF-16 code point", "USV", "byte", etc. "Character" leaves far too much up to the imagination of the reader.

No, codepoints are, hence their name. Scalars are a subset of all codepoints. https://stackoverflow.com/questions/48465265/what-is-the-dif...

> whether that's "UTF-16 code point"

That's not a thing; you're thinking of UTF-16 code units rather, I believe.

Yes, yes, the `str` type may contain data that doesn't represent a valid string. I've already explained elsewhere ITT that this is a feature.

And sure, pedantically it should be "UCS-4" rather than UTF-32 in my post, since a str object can be created which contains surrogates. But Python does not use surrogate pairs in representing text. It only stores surrogates, which it considers invalid at encoding time.

Whenever a `str` represents a valid string without surrogates, it will reliably encode. And when bytes are decoded, surrogates are not produced except where explicitly requested for error handling.

> The number of Unicode scalar values in the string. (If the string were encoded in UTF-32, the length of that array.)

Ah.

Good news: since Python doesn't use surrogate pairs to represent valid text, these are the same whenever the `str` contents represent a valid text string in Python. And the cases where they don't, are rare and more or less must be deliberately crafted. You don't even get them from malicious user input, if you process input in obvious ways.

> The Unicode definition of "character" is not a technical definition, it's just there to help humans.

You're missing the point. The facepalm emoji has 5 characters in it. The Unicode Consortium says so. And they are, indisputably, the ones who get to decide what a "character" is in the context of Unicode.

I linked to the glossary on unicode.org. I don't understand how it could get any more official than that.

Or do you know another word for "the thing that an assigned Unicode code point has been assigned to"? cf. also the definition of https://www.unicode.org/glossary/#encoded_character , and note that definition 2 for "character" is "synonym of abstract character".

I just relied on this fact yesterday, so it's kind of a funny timing. I wrote a little script that looks out for shenanigans in source files. One thing I wanted to explore was what Unicode blocks a given file references characters from. This is meaningless on the byte level, and meaningless on the grapheme cluster level. It is only meaningful on the codepoint level. So all I needed to do was to iterate through all the codepoints in the file, tally it all up by Unicode block, and print the results. Something this design was perfectly suited for.

Now of course:

- it coming in handy once for my specific random workload doesn't mean it's good design

- my specific workload may not be rational (am a dingus sometimes)

- at some point I did consider iterating by grapheme clusters, which the language didn't seem to love a whole lot, so more flexibility would likely indeed be welcome

- I am well and fully aware that iterating through data a few bytes at a time is abject terrible and possibly a sin. Too bad I don't really do coding in any proper native language, and I have basically no experience in SIMD, so tough shit.

But yeah, I really don't see why people find this so crazy. The whole article is in good part about how relying on grapheme cluster semantics makes you Unicode version dependent and that being a bit hairy, so it's probably not a good idea to default to it. At which point, codepoints it is. Counting scalars only is what would be weird in my view, you're "randomly" doing skips over the data potentially.

Also good against data fingerprinting, homoglyph attacks in links (e.g. in comments), pranks (greek question mark vs. semicolon), or if it's a strictly international codebase, checking for anything outside ASCII. So when you don't really trust a codebase and want to establish a baseline, basically.

But I also included other features, like checking line ending consistency, line indentation consistency, line lengths, POSIX compliance, and encoding validity. Line lengths were of particular interest to me, having seen some malicious PRs recently to FOSS projects where the attacker would just move the payload out of sight to the side, expecting most people to have word wrap off and just not even notice (pretty funny tbf).