Yeah, you wouldn't be using 'kjempestort' when writing a scientific paper. Though one could argue that the use of 'veldig' and 'very' should be kept to a minimum in that setting as well.
However, I don't think of it as too casual in day-to-day language. Just follow the same rule as with 'very': Don't overuse it.
While 'kjempestort' and 'veldig stort' both can translate to 'very big', 'kjempestort' can also translate to 'huge', 'enormous', etc... it's a potentially stronger adjective if used sparingly, but at the same time it's often used in exaggerations - so take it with a grain of salt when other people use it.