On 2021-02-05 16:25:09 -0800 (-0800), Clint Byrum wrote: [...]

I paused zuul dev in a place where pure vim with almost no plugins or config were how I developed Python, and I didn't really understand type hinting other than "yay another way to prevent bugs". And it *is* very much that, and it will catch some bugs before they happen at runtime.

But since doing moving on to other things, I've adopted VS:Code for python dev, and I have to say, it helps *a lot* with reducing the cognitive load of writing and editing code. Just being able to type an object's name, and then hit ., and getting a menu of the methods avaliable, means I never accidentally try to call .split on a list, or pass a string without first casting to the numeric type required by a function, thus locating the error handling near the ingest of the data, rather than the use of the number.

[...]

Thanks, this is an interesting perspective. I've honestly always wrestled with IDEs, and chucked more than one VIM plugin which implemented hacky sorts of tab completion for Python source, but if you're finding Versus Code to be helpful, I'll try to remember to find it and take it for a spin soon.

With all due respect but I think this would be a big step backwards.

Yes, it is true that type annotations increase verbosity of the code and are hard to get right and that mypy has bugs.

Still, type hints do document the code much better than docstrings and force developer to write better code from start.

Since I started to use type annotation less than two years ago and slowly introduce it to some projects, I learned a lot from it. I would go so far to say that type hints "trained" me to use less magic in Python code and this translated to more reliable and easier to maintain code, safer.

I observed that despite being more verbose, it is much easier to make a contribution to a random python project that has type hints, because your code editor/IDE will help you considerably when it has information about type, something that is not possible without. Without type information code is much harder to touch as it requires a lot of investigation, even for minor changes.

Assume you have a function receives an argument foo that "looks to be string", based on function body and you want to add something to it. You may easily introduce a bug if out of 10 places where is called from, one is passing something else, that would break with your minor change. Unitests and reviews will likely not prevent you from introducing a regression here, but having a type will likely do spot it.

There were several cases where typing by itself uncovered bugs in the code, things that passed unittesting and human reviewers.

As I am actively involved with many upstream python libraries, I can see that there is clear trend of migrating towards type-hints, what did prevented them for moving sooner was the fact that typehints before python 3.6 were a PITA (i would not recommend them to anyone).

One notable use of typehints is the Pydantic https://pydantic-docs.helpmanual.io/ project which is a library that can generate JSON/YAML schema validation for configuration files based on the type annotations.

This library could be used to generate a schema for our zuul job config files, something that is extremely hard to write by hand using the specification https://json-schema.org/.

I am really curious to see which notable projects made the decision to drop type hints?

I do strongly believe that the future of Python includes more types not less, as it did with JavaScript/TypeScript.

What makes Zuul so unique in that regard that we see type information as a bad thing?

Best regards Sorin Sbarnea

On Fri, Feb 05, 2021 at 17:07 James E. Blair wrote:

Clint Byrum clint@fewbar.com writes:

...

Hi! I'm on hiatus from Zuul dev as it's not in any of my current day-job stacks nor do I have spare time to work on it. So take my words as perspective, not as any sort of vote either way.

Thanks for your perspective. This is going to be a tricky discussion. I doubt we will all agree 100% on an answer, so this is going to be an exercise in finding consensus. I do think we need to hear from different perspectives and I appreciate your presenting those of junior developers (especially since they may not feel like speaking up). Ultimately we're going to have to make a decision that not everyone is going to be happy with, so I ask everyone to keep that in mind. Consensus in this case is likely to be "something that we can all live with".

...

On Fri, Feb 5, 2021 at 1:54 PM James E. Blair jim@acmegating.com wrote:

...

Hi,

I'd like to propose that we remove type annotations and checking from our code.

I don't think they are providing much value, but they come at a significant cost.

They have very rarely (I think perhaps once or twice) caught errors. Meanwhile, we aren't going to relax our test coverage requirements, which should catch most things anyway.

I don't believe that this is all that telling. Running the type checker in CI is meant to ensure it happens, in case someone forgets. It's not meant to be the place at which type violations are detected and dealt with.

Neither do I think that running it just before commit is the general way it is most useful, but that should beh appening, and that would be a thing that happens well outside the eyes of anyone other than the developer writing a change, so it's hard to really measure how useful some find it.

Well, I mention it because that was certainly the argument made to begin the experiment in the first place. It's possible that the field has shifted since then though, and the value lies elsewhere as you say.

It can be tricky to ensure mypy do check the annotations and that is a bit concerning since it may accept incorrect annotations. When mypy does check the annotations, it uses very permissive coercion rules by default, which can also result in valid incorrect annotations. For example it is tedious to track Any type inference.

I find it frustrating when mypy accepts invalid code, even after adding all the necessary annotations.

...

However, the more typed core models and libraries are, the more editor plugins will be able to guide a developer to the right answer.

I paused zuul dev in a place where pure vim with almost no plugins or config were how I developed Python, and I didn't really understand type hinting other than "yay another way to prevent bugs". And it *is* very much that, and it will catch some bugs before they happen at runtime.

But since doing moving on to other things, I've adopted VS:Code for python dev, and I have to say, it helps *a lot* with reducing the cognitive load of writing and editing code. Just being able to type an object's name, and then hit ., and getting a menu of the methods avaliable, means I never accidentally try to call .split on a list, or pass a string without first casting to the numeric type required by a function, thus locating the error handling near the ingest of the data, rather than the use of the number.

I know, experienced devs aren't going to care as much, but I want to suggest that type hinting is a rather pleasant and normal thing for some more junior developers.

For what it's worth, that's not the first time I've heard this, and I am sympathetic to the argument. I do think it's worth considering.

Note that language server protocol (LSP) also works well with neovim and emacs where you can get a similar experience. I'm using lsp-mode to find definitions and to autocomplete attributes.

...

...

On the other hand, they cause the code to be much less readable and I believe that will ultimately lead to more errors. Developer and reviewer fatigue are important things to keep in mind. Some examples where I think annotations are particularly unhelpful:

self.on_connect_listeners: List[Callable[[], None]] = []

Or (note the duplication):

def __init__(self, ... info: Optional[zuul.model.WebInfo] = None): self.info: Optional[zuul.model.WebInfo] = info

We've also had some occurrences where we ended up fighting the annotation system.

This is definitely a real thing. Mypy has gone through a few cycles now where they've clearly painted themselves into corners and there are too many escape hatches, so folks aren't fixing the hard bugs.

I agree that what we see in practice does not match what we all hope for from mypy.

I think these issues are significant enough that, along with the fact that it's an appendage to the language, it makes the resulting code very difficult to read and understand. That is something that affects all developers, junior and senior.

I've studied that topic quite a bit recently. Type systems effectively prevents the occurrence of execution errors and can exclude certain classes of programming errors. One of the benefits is that it reduces testing burden and QA can focus on higher impact bugs.

This lead me to learn Haskell, OCaml and Rust where types are richer. I now find the Python type annotations, and the mypy checker too limited for the following reasons:

- The Any type removes all guarantees. - Pattern matching on unions are not supported.

For example, an annotated `Build` object would looks like this:

``` # zuul/model.py class Build(object): def __init__(self, job: Job, uuid: str): self.job = job self.uuid = uuid self.url: Optional[str] = None self.result: Optional[str] = None self.result_data: Dict[str, Any] = {} self.error_detail: Optional[str] = None ... ```

Which has the following issues:

- A SUCCESS build may still have a missing url. - The result data is defined for build without result (e.g. NODE_FAILURE).

It is nice that mypy will force us to check if the url is defined before using it, but it would be nicer if we could annotate that `url` is only valid for SUCCESS or FAILURE result, and not available for NODE_FAILURE result.

With pattern match, the Build type could looks like this:

``` type Build = Success(url, result_data) | Error(url, error_detail) | NodeFailure

def show_build(build : Build) -> str: match build: Success(url, result_data): return url Error(url, error_detail): return url NodeFailure: return "node failure" ```

The zuul model currently encodes a lot of different states in general purpose object attributes, for example `QueueItem` or `Change`. I think that annotating these objects will clutter the definition with Optionals, and the type checker will require many un-necessary tests when using these objects.

Thus I think it will be a significant effort to annotate the Zuul code base, and still, we will be missing a lot from a richer type system. I recommend this Jane Street's talk about what an expressive type systems can do: https://www.youtube.com/watch?t=961&v=kZ1P8cHN3pY

...

...

I've mentioned to several of you that I do very much enjoy the benefits of statically typed languages (I write C code for fun), but I don't think that the type hints in Python are very well executed. Many of us chose to use Python instead of Java specifically for the dynamic typing and as a reaction against the extensive type requirements. The type hinting that is being added to Zuul seems like it carries the costs of Java without the benefits.

I think Python's optional typing is far simpler than the required, explicit, verbose typing of java. The comparisons to C is completely irrelevant IMO: it is a low level language where the types are more indicative of storage than utility.

I didn't think the mention of C was irrelevant, I do understand the technical differences here. I just wanted to establish this wasn't some arbitrary vendetta I have against typed languages. But this is a meta-argument so let's just leave it.

And yes, Python's typing is optional, but this discussion is explicitly about using it, and not just using it a little around the edges, but really committing to it. Effectively using it in a non-optional way.

I think the main difference is that Python annotations can be inferred: you don't have to declare an object struct outright, and assignments automatically get the return type. In theory, function's argument types could also be infered in some case, but Python uses Any.

...

...

We started this as a small experiment. We said that we would run the type checker in a mode where it would operate on implicit type information, thereby minimizing the additional cognitive load on authors and reviewers. We then started adding some minimal type information in strategic places to help it out. However, I'm now reviewing changes that are literally 3x larger than they otherwise would be because of all the additional repetitive type information going into them.

Though we could use type aliases to reduce the repetition.

...

...

I'm glad to try experiments like these, but I think this one has grown to the point where we need to decide if we want to go all-in on type annotations or remove them.

I feel fairly strongly that the cost to authors and reviewers far outweighs the benefits, and we should make everyone's lives easier by removing them.

Again, I'm not an active developer or reviewer, so it's just a thought. But make sure you hear from more junior contributors before determining the value.

It seems like we would need to add a lot more annotations to have useful mypy checks. Though if the existing annotations can help developper using IDE support, then I do not mind keeping what we already have.

If we decide to go all-in on type annotations, I would consider waiting for pattern matching support in Python. That would be a good opportunity to isolate object state in dedicated type instead of the current definitions that relies on object attributes value. I think this has the potential to significantly improve the Zuul code base quality.

Regards, -Tristan

On Sun, Feb 07, 2021 at 17:42 Jeremy Stanley wrote:

On 2021-02-07 16:52:20 +0000 (+0000), Tristan Cacqueray wrote: [...]

...

If we decide to go all-in on type annotations, I would consider waiting for pattern matching support in Python. That would be a good opportunity to isolate object state in dedicated type instead of the current definitions that relies on object attributes value. I think this has the potential to significantly improve the Zuul code base quality.

As in PEP 642 and its predecessors/alternatives? That does look interesting, and I'm interested to see which way the steering council splits on the topic. Last committer poll I saw, there was still a substantial (albeit minority) contingent who didn't want pattern matching in the language at all.

Yes, this PEP or one of the alternatives would be ideal. PEP 0604 is also going in that direction. Hopefully, mypy would then properly support such union with exhaustiveness check.

-Tristan

On 2021-02-08 09:38:02 -0800 (-0800), Clark Boylan wrote: [...]

While I'm not a huge fan of the verbose and sometimes redundant syntax for type checking, I think what bothers me most is that when we want to update Zuul's python platforms we end up debugging mypy and turning off checks. I know we don't have a ton of control over the mypy tool itself, but I wonder if we could be approaching this in a better way somehow. Does mypy have a way to opt in to new checks rather than opt out? or maybe we only run mypy on a fixed version of python and do upgrades less frequently?

Just throwing this out there as a frustration and maybe others know of ways to handle it better.

[...]

Over the weekend I did a bit of digging into the current state of type hinting, and it looks like Python 3.9 added a lot of simplification. typing.Optional lets you toss a lot of those nasty Null unions, more of the stdlib natively supports type arguments now, et cetera. If we could somehow switch to just focusing on 3.9's type hinting and not support type hints for earlier interpreters, as a compromise, it might not be nearly so onerous.

On 08.02.21, 23:27, "James E. Blair" jim@acmegating.com wrote: (...) I'm not [yet] an expert on type annotations in Python, so I may have to rely on others who know more details on what we can and should do in the future, but here are some guidelines that might be a good starting point:

* Let's not bend over backwards to make mypy happy. That's not our goal, so if something is tripping it up but is otherwise not ambiguous to humans or IDEs, do the minimum work to ignore it.

* Corollary: do we actually need/want to keep running mypy automatically in our Zuul jobs?

* Let's focus on improving interfaces, but not exhaustively typing every variable.

* If there are improvements coming in the future (like 3.9) let's do less now and wait for them.

And based on those, what do you think about these specific guidelines:

* Do type method signatures. * Don't recurse ("List[Callable]", not "List[Callable[[], None]]"). * Avoid typing variable assignment inside of methods. (They should mostly be inferred, and they do little to improve code comprehension.)

-Jim

I like those guidelines. They seem to be a good way to have the advantages of typings while avoiding most of the drawbacks that have been mentioned in the initial post. I think we may want to allow exceptions to the don't recurse rule if they are needed/useful. In this case however I think we should alias them to keep the method signatures readable.

I also agree that we should try to avoid typings of local variables since they (at least for me personally) have the most effect on the readability of the code.

I think there is a further guideline we could think of: * Don't mix adding annotations to existing code and functional changes if a large part of the change is adding annotations. Such changes should be split into a non-functional change and a functional change.

Regarding mypy I think we should keep running it and maybe switch it to non-voting in case it keeps popping up with problems so issues with it can be fixed asynchronously without blocking other development.

- Tobias

Tobias.Henkel@bmw.de writes:

On 08.02.21, 23:27, "James E. Blair" jim@acmegating.com wrote: (...) I'm not [yet] an expert on type annotations in Python, so I may have to rely on others who know more details on what we can and should do in the future, but here are some guidelines that might be a good starting point:

* Let's not bend over backwards to make mypy happy.  That's not our
  goal, so if something is tripping it up but is otherwise not ambiguous
  to humans or IDEs, do the minimum work to ignore it.

  * Corollary: do we actually need/want to keep running mypy
    automatically in our Zuul jobs?

* Let's focus on improving interfaces, but not exhaustively typing every
  variable.

* If there are improvements coming in the future (like 3.9) let's do
  less now and wait for them.

And based on those, what do you think about these specific guidelines:

* Do type method signatures.
* Don't recurse ("List[Callable]", not "List[Callable[[], None]]").
* Avoid typing variable assignment inside of methods.  (They should
  mostly be inferred, and they do little to improve code comprehension.)

-Jim

I like those guidelines. They seem to be a good way to have the advantages of typings while avoiding most of the drawbacks that have been mentioned in the initial post. I think we may want to allow exceptions to the don't recurse rule if they are needed/useful. In this case however I think we should alias them to keep the method signatures readable.

I also agree that we should try to avoid typings of local variables since they (at least for me personally) have the most effect on the readability of the code.

I think there is a further guideline we could think of:

• Don't mix adding annotations to existing code and functional changes if a large part of the change is adding annotations. Such changes should be split into a non-functional change and a functional change.

Regarding mypy I think we should keep running it and maybe switch it to non-voting in case it keeps popping up with problems so issues with it can be fixed asynchronously without blocking other development.

That sounds good. I think we can probably start incorporating these guidelines into our habits at this point, without being too fussy about this.

Thanks to everyone who chimed in on this thread. I learned some things and I'm happy with the conversation and the result.

-Jim