Just incase anyone struggles as I did, you can have a div position to fixed as well as relative. Let's call this div, "div A" that way the divs that you have within your fixed div aka "div A" which you make absolute so they relate to "div a" which is relative.
Just a recap
Div a - position - fixed
Position - relative
The divs that you want to be placed within that fixed div