The "monsters" featured in this image appear as larger and exaggerated versions of known creatures. My imagination is unconvinced that they are actually below the surface of the water over which graceful dolphins dive and on which the monks' modest boat floats. My brain tells me this is a mashup of two different pages in the story. These creatures might be inspired by nature, however they exist not in the literal world, but rather in the sphere of human perception.
Robert Gibbings was a founding and key member of the Society of Wood Engravers. More on Saint Brendan the Navigator here.