Here's a few to try, in no particular order:
1) Belleayre park (30 minutes from Woodstock)
2) Saugerties Village Beach (30 minutes from Woodstock)
3) Kingston Point Beach (25 minutes from Woodstock)
4) North South Lake (40 minutes from Woodstock) https://www.dec.ny.gov/outdoor/24487.html
Please note, I have not tried any of the lake beaches personally. I would recommend calling the numbers of the websites to verify the information is up-to-date.