For crystallography specifically, there's ourselves (Materials Project), OQMD, AFLOW, Materials Cloud, JARVIS, and a number of more specific (but no less important) specialized databases. There are also a number of commercial offerings.

Best practices are incredibly difficult. We're trying to establish a common API currently (https://github.com/Materials-Consortia/optimade) that can be adopted by all database providers. How the data is stored behind the scenes is something that ends up being very specific to how the data is generated and what its applications are. We're definitely better as a community than we were ten years ago, but there's a lot of work to be done here.

In terms of scientific databases outside crystallography/materials science, Nature's Scientific Data is a good open-access journal to peruse: https://www.nature.com/sdata/

NCBI's Gene Expression Omnibus (GEO) for bulk and single-cell RNA sequencing

https://www.ncbi.nlm.nih.gov/geo/

The Cancer Genome Atlas (TCGA) for a wide-array of data from cancer patients in the US (Treatment history, outcomes, histology, expression, etc)

https://portal.gdc.cancer.gov/

InterPro from EMBL harmonizes multiple protein family databases

https://www.ebi.ac.uk/interpro/

STRING harmonizes protein-protein interaction databases

https://string-db.org/

There's the classic, Protein Data Bank. A number of people in the field saw it as a moral imperative. It's interesting how open different disciplines are to openness