The IRC and IBC do specify minimum sizes, so if your county adopted either of those then there is a built in minimum.

> Every dwelling unit shall have at least one habitable room that shall have not less than 120 square feet (11 m2) of gross floor area. Other habitable rooms shall have a floor area of not less than 70 square feet (6.5 m2). Habitable rooms shall not be less than 7 feet (2134 mm) in any horizontal dimension.

The 2015 revision removes or modifies this.