First up, don’t eat the penguins or shearwaters. They are protected. They also taste awful (so I am told).
The easiest way you can get together some seafood, without even leaving dry land, is to forage for mussels at low tide. At the western end of the island there are masses of the things, but please be careful as the rock shelf is exposed to waves. Only suitable to go at low tide on calm days.
Next easiest is to throw in a line. Squid can be caught casting out and winding in a squid jig. Flathead can be hooked from the sandy bottom if you cast out a bit too. For a bit of fun (but not great eating) you can drop a line straight down off the jetty for parrotfish and leatherjackets.
For divers there are sea urchins readily collected at shallow depths (although I can’t see the attraction in eating them, some love them). Abalone are also out there, although you will have to dive down and search within the seaweed. Crayfish can also be found if you are into swimming head first into tiny caves on a single breath of air.
Crayfish and abalone require a fishing licence and there are bag limits (and yes inspectors have been known to check so please make sure you get licensed at Service Tas and make sure the season is open). You don’t need a licence for rod fishing in Tasmania.
If all this sounds a bit hard, buy some live oysters from the Freycinet Marine Farm and stick them in a rockpool. You will feel like Bear Grylls when you wander down to the sea, sauvignon blanc in hand and shuck them on a rock.