A man was swept away while fishing and dozens of other people were injured as strong winds and rain battered the island.