hddqsb

I had a look at the demo (https://huggingface.co/spaces/TabbyML/tabby) and wasn't too impressed with the generated code for the default sample prompt (binary search) -- it recurses infinitely if the item is missing. It would be interesting to compare with Copilot's output. No idea how one would go about fixing this (other than manually add a correct binary search implementation to the training data, which feels like cheating).

Request to https://tabbyml-tabby.hf.space/v1/completions:

    {
      "language": "python",
      "prompt": "def binarySearch(arr, left, right, x):\n    mid = (left +"
    }

    {
      "id": "cmpl-...",
      "created": 1680867355,
      "choices": [
        {
          "index": 0,
          "text": " right) >> 1\n    if x < arr[mid]:\n        return binarySearch(arr, left, mid - 1, x)\n    elif x > arr[mid]:\n        return binarySearch(arr, mid + 1, right, x)\n    else:\n        return mid"
        }
      ]
    }

    def binarySearch(arr, left, right, x):
        mid = (left + right) >> 1
        if x < arr[mid]:
            return binarySearch(arr, left, mid - 1, x)
        elif x > arr[mid]:
            return binarySearch(arr, mid + 1, right, x)
        else:
            return mid

   arr = [1, 3, 5, 7]
   print(binarySearch(arr, 0, len(arr), 5)) # 2 (correct)
   print(binarySearch(arr, 0, len(arr), 4)) # RecursionError

https://tio.run/##lY/BCoJAEIbvPsUPXVw0yCyCKG@9QB3Fg@maC7bKuI...

Since GitHub's IP address is not stable, I suggest disabling the IP-checking feature using

    Host github.com
         CheckHostIP no

The PDF has been updated to address that:

> Can someone else “squat” my namespace?

> No. Even if your organization is suspended, deleted, or you choose to leave Docker voluntarily, your organization’s namespace will not be released, so other users can’t “squat” your images.

See Jonathan Leffler's comment on SO (also demonstrated by the the top-level parent comment here):

> As noted in the answer by kojiro, it is perfectly possible and legal to start the pipeline with `< file command1 ...`. Although the conventional position for the I/O redirection operators is after the command name and its arguments, that is only the convention and not a mandatory placement. The `<` does have to precede the file name. So, there's a close to perfect symmetry between `>output` and `<input` redirections: `<input command1 -opt 1 | command2 -o | command3 >output`.

Here are the videos:

- “Loki: A unified multiphysics simulation framework for production,” presented at SIGGRAPH 2022 https://www.youtube.com/watch?v=_2TBvSnsLD8

- “Wave Curves: Simulating Lagrangian water waves on dynamically deforming surfaces,” presented at SIGGRAPH 2020 https://www.youtube.com/watch?v=FqCK5Gu-Op4

- Overview of “Guided bubbles and wet foam for realistic whitewater simulation” as presented at SIGGRAPH 2022. Video courtesy of Wētā FX. https://www.youtube.com/watch?v=60Dk5h_659s

Code:

    for (const deck of __NEXT_DATA__.props.pageProps.entity.decks) if (deck.url) console.log(deck.caption, deck.url)

> Your code is probably saved in UTF-8 right now even if you don't use non-ASCII characters.

Nit: For a document that only contains ASCII characters, UTF-8 is identical to ASCII. So calling it UTF-8 is technically correct, but misleading -- you could equally well say the document is encoded using any of the other supersets of ASCII.

Ha ha, I guess footgun sizes are all relative. The quirky error handling of sh is "well-known" (usually one of the first pieces of advice given to improve safety is to insert `set -e` at the top of every shell script, which mostly fixes this issue). So I don't think of Dockerfile heredocs themselves as a large footgun, but rather as a small footgun that arises out of the small interaction between heredocs and the large-but-well-known error handling footgun.

I don't know why Docker doesn't use `set -e` by default. I suppose one reason is for consistency -- if you have shell commands spread across both a Dockerfile and standalone scripts, it could be very confusing if they behaved differently because the Dockerfile uses different defaults.

I also don't know why the commands are coupled to the layers. Maybe because in the simple cases, that is the best mapping; and in the very complex cases, the commands would be moved to a standalone script; so there are fewer cases where a complex command needs to be inlined into the Dockerfile in a way that produces a single layer.

It would be really nice if the Dockerfile gave more control over layers. For example, currently if you use `COPY` to import files into the image and then you use `RUN to you modify them (e.g. to change the ownership / permissions / timestamps), it would needlessly increase the image size; the only way to avoid this is to perform those changes during the COPY, for example using `COPY --chown`; but COPY has very limited options (namely: chown, and also chmod but that is relatively recent).

Regarding native support for lists of commands, I don't really see much value since sh already supports lists (you "just" need to correctly choose between "&&" and ";"/newline).

> > This allows you to easily create a heredoc containing other heredocs.

> Hmm, what’s the use-case for that? The only effect for the programmer would be to change the escape sequence, no?

It can be useful to embed entire files within a script (e.g. when writing a script that pre-populates a directory with some small files). With most quoting schemes, you'd have to escape special characters that appear in those files. But with heredocs, you just have to pick a unique token and then you can include the files verbatim.

(Picking a token that doesn't appear as a line within the files can be a little tricky, but in many cases it's not a problem; for example if the files to be included are trustworthy, it should be enough to use a token that includes the script's name. On the other hand if the data is untrusted, you'd have to generate an unguessable nonce using a CSPRNG. But at that point it's easier to base64-encode the data first, in which case the token can be any string which never appears in the output of base64, for example ".".)

> > Nice syntax

> Is it though?

Before the heredoc syntax was added, the usual approach was to use a backslash at the end of each line, creating a line continuation. This has several issues: The backslash swallows the newline, so one must also insert a semicolon* to mark the end of each command. Forgetting the semicolon leads to weird errors. Also, while Docker supports line continuations interspersed with comments, sh doesn't, so if such a command contains comments it can't be copied into sh.

The new heredoc syntax doesn't have any of these issues. I think it is infinitely better :)

(There is also JSON-style syntax, but it requires all backslashes to be doubled, and is less popular.)

*In practice "&&" is normally used rather than ";" in order to stop the build if any command fails (otherwise sh only propagates the exit status of the last command). This actually leads to a small footgun with the heredoc syntax: it allows the programmer to use just a newline, which is equivalent to a semicolon and means the exit status will be ignored for all but the last command. The programmer must remember to insert "&&" after each command, or use `set -e` at the start of the RUN command, or use `SHELL ["/bin/sh", "-e", "-c"]` at the top of the Dockerfile. But this footgun is due to sh's error handling quirks, not the heredoc syntax itself.

> Are the line breaks semantic, or is it all a multiline string?

The line breaks are preserved ("what you see is what you get").

> Is EOF a special end-of-file token

You can choose which token to use (EOF is a common convention, but any token can be used). The text right after the "<<" indicates which token you've chosen, and the heredoc is terminated by the first line that contains just that token.

This allows you to easily create a heredoc containing other heredocs. Can you think of any other quoting syntax that allows that? (Lisp's quote form comes to mind.)

> Where is it documented?

The introduction blog post has already been linked. The reference documentation (https://docs.docker.com/engine/reference/builder/, https://github.com/moby/buildkit/blob/master/frontend/docker...) explains the syntax using examples. It doesn't have a formal specification; unfortunately this is a wider problem with the Dockerfile syntax (see https://supercontainers.github.io/containers-wg/ideas/docker...). Instead, the reference links to the sh syntax specification (https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V...), on which the Dockerfile heredoc syntax is based.

The argument boils down to "here is a bad thing about raising capital, here is a good thing about not raising capital, therefore not raising capital is better".

But there are other pros and cons that this argument is ignoring, so it is not logically correct.

I also think the version in the article is an unfair representation of the original blog post (https://ensorial.com/2020/dont-raise-money/), which explicitly says

> I’m not suggesting that there are never reasons to take outside investment. Obviously there are. But, we should recognise that doing so comes with significant trade-offs and difficulties that mean it shouldn’t necessarily be the default.

(I see that @ilyt already posted this exact quote.)

The C23 change was mostly to downgrade some of the existing reserved identifiers from "reserved" to "potentially reserved". (It also added some new reserved and potentially reserved identifiers, but they seem reasonable to me.)

POSIX reserves "_t" suffix everywhere (not just for identifiers beginning with "int"/"uint" from <stdint.h>); references: https://www.gnu.org/software/libc/manual/html_node/Reserved-..., https://pubs.opengroup.org/onlinepubs/9699919799/functions/V....

So I actually stand by my original comment that the convention of using "_t" suffix shouldn't be recommended. (It's just that the reasoning is for conformance with POSIX rather than with ISO C.)

> The goal of the future language and library reservations is to alert C programmers of the potential for future standards to use a given identifier as a keyword, macro, or entity with external linkage so that WG14 can add features with less fear of conflict with identifiers in user’s code. However, the mechanism by which this is accomplished is overly restrictive – it introduces unbounded runtime undefined behavior into programs using a future language/library reserved identifier despite there not being any actual conflict between the identifier chosen and the current release of the standard. ...

> Instead of making the future language/library identifiers be reserved identifiers, causing their use to be runtime unbounded undefined behavior per 7.1.3p1, we propose introducing the notion of a potentially reserved identifier to describe the future language and library identifiers (but not the other kind of reservations like __name or _Name). These potentially reserved identifiers would be an informative (rather than normative) mechanism for alerting users to the potential for the committee to use the identifiers in a future release of the standard. Once an identifier is standardized, the identifier stops being potentially reserved and becomes fully reserved (and its use would then be undefined behavior per the existing wording in C17 7.1.3p2). These potentially reserved identifiers could either be listed in Annex A/B (as appropriate), Annex J, or within a new informative annex. Additionally, it may be reasonable to add a recommended practice for implementations to provide a way for users to discover use of a potentially reserved identifier. By using an informative rather than normative restriction, the committee can continue to caution users as to future identifier usage by the standard without adding undue burden for developers targeting a specific version of the standard.

The purpose of this particular naming rule is to allow adding new typedefs such as int128_t. The "undefined behaviour" part is for declaration of any reserved identifier (not specifically for this naming rule). I don't know why the standard uses "undefined behaviour" instead of the other classes (https://en.cppreference.com/w/cpp/language/ub); I suspect because it gives compilers the most flexibility.

> Here are the absolute essential flags you may need.

I highly recommend including `-fsanitize=address,undefined` in there (docs: https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.h...).

(Edit: But probably not in release builds, as @rmind points out.)

> The closest thing to a convention I know of is that some people name types like my_type_t since many standard C types are like that

Beware that names beginning with "int"/"uint" and ending with "_t" are reserved in <stdint.h>.

[Edited; I originally missed the part about "beginning with int/uint", and wrote the following incorrectly comment: "That shouldn't be recommended, because names ending with "_t" are reserved. (As of C23 they are only "potentially reserved", which means they are only reserved if an implementation actually uses the name: https://en.cppreference.com/w/c/language/identifier. Previously, defining any typedef name ending with "_t" technically invokes undefined behaviour.)"]

The post never mentions undefined behaviour, which I think is a big omission (especially for programmers coming from languages with array index checking).

> void main() {

As @vmilner mentioned, this is non-standard (reference: https://en.cppreference.com/w/c/language/main_function). The correct declaration is either `int main(void)` or the argc+argv version.

(I must confess that I am guilty of using `int main()`, which is valid in C++ but technically not in C: https://stackoverflow.com/questions/29190986/is-int-main-wit...).

> You can cast T to const T, but not vice versa.

This is inaccurate. You can implicitly convert T* to const T*, but you need to use an explicit cast to convert from const T* to T*.

    struct tmp* m_result_original=...;
    struct tmp* m_result_my_version=...;
    ... memcmp(&m_result_original, &m_result_my_version, ...)

Ironically, GCC 12.2 warns about this error while clang doesn't :)

The warning message produced by GCC is "'memcmp(...)' specified bound 12 exceeds source size 8 [-Wstringop-overread]", and it is printed even without -Wall, but only when the third argument is `sizeof(struct tmp)` rather than `sizeof(struct tmp)!=0`.

But as @Stratoscope pointed out, memcmp() is still problematic if the struct has padding.

With -O2: https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(filename...

Without -O2: https://godbolt.org/#g:!((g:!((g:!((h:codeEditor,i:(filename...

[Edited. I originally wrote "It should be noted that this only happens when..." but inserted "in this particular case" after @eru pointed it out.]

[1]: https://www.gnu.org/software/emacs/manual/html_node/elisp/Lo...

[2]: https://www.gnu.org/software/emacs/manual/html_mono/cl.html#...