JSFiddle - React, Tailwind, and code Playground

HTML

<body>
  <header> Header content, might contain an image</header>
  <section> This is the sub header, unknown height </section>
  <div class="container">
    <div class="fill">I Want</div>
    <div class="fill">Each of These</div>
    <div class="fill">To be </div>
    <div class="fill">The height of the body - the Header - the Section</div>
  </div>
</body>

CSS

body {
  display: flex;
  flex-direction: column;
  height: 100vh;
  margin: 0;
}

.container {
  flex: 1;
  display: flex;
  flex-direction: column;
  overflow-y: scroll;
}

.fill { flex: 0 0 100%;
}

header  { background-color: aqua; }
section { background-color: orange; }
.fill:nth-child(odd) { background-color: yellow; }
.fill:nth-child(even) { background-color: lightgreen; }