Pydantic - how it can be used to fill in the void of Hydra. Hydra - its strengths and also, what I think are its weakness.The structure of this post is as follows: Lastly, put forth a tentative solution for this gap. In this post, I will go over my learnings of both the tools and also talk about gaps I still see present. With that in mind, I evaluated a bunch of tools combination of Hydra and Pydantic stood out as a good choice. A configuration management that nicely integrates into your echo-system of tools and is not very dictative.Ability to integrate easily into complex distributed launcher frameworks for both local and remote launch.Ability to version config and enforce a schema on the config specification for reliability and robustness.This perhaps is optional if one is not interested in dynamic domain-specific validations - something that can not be achieved in serialized formats like yaml/json. Support structured config consisting of potentially complex, canonical python types with strong validation support beyond type hint.Being able to store and reuse the final assembled/compiled config and reproduce the experiments reliably.Ability to compose config using templates and applying customization through overrides, environments etc.Without duplication, reuse configurations across modules wherever needed by means such as interpolation, expressions.Ability to organize the configuration of training in a hierarchical fashion acknowledging the dependency and hierarchy of participating components.Ability to easily templatize configurations of various components of your training experiments.Overall, configuration solution for training experiments has following requirements: A classifier network needs to output a 10 length vector if the data source is MNIST and 20 if it’s PASCAL VOC. model’s output specification needs to be in line with data specification. While these are loosely coupled specifications, they may enforce some degree of restrictions on each other e.g. For example, an experiment configuration can be broken down into data and model specifications. How well does this integrate with other ecosystem toolsīy definition model training configurations are hierarchial.Some other interesting features of hydra.This can get real messy real quick with deep learning experiments where the list of config and hyperparameters can rapidly grow out of hand as soon as one breaks out of adhoc experimentations. One key differentiator perhaps here is how many config fields one has and how each of these relates to the other. This is true for any software application however some applications demand higher investment on the config piece than others. Some of the indirect advantages of having a good config solution are clean, reliable, and simple application code. How do you manage the configurations of your model training experiments? Having good configuration management in place improves the user experience and simplifies experiment management.
0 Comments
Leave a Reply. |