On paper, SCotUS _can_ be held accountable by the public through their representatives. The process is called impeachment. The house _can_ impeach a sitting Justice, and the Senate _can_ vote to remove them from office. This currently is unlikely, but the it _is_ possible.

The "easier" route is for the legislature to "see" the ruling and pass a law that counter-acts it. The general public can _also_ call their own Constitutional conventions and pass amendments to the Constitution to hold SCotUS responsible.

The real "problem" is all of these solutions, by design, require significant effort. If you want changes, you will need to make them happen.