Born in Herndon, Virginia, in 1864, Castleman was the daughter of an Episcopal clergyman and a teacher who established the Herndon Seminary for Girls.  After attending the Edgeworth Institute in Baltimore, Virginia Carter Castleman arrived at Drexel in 1898 at the age of 34.  Little is known of her life prior to her arrival in Philadelphia.  While attending Drexel she wrote the words for the Drexel Ode.  After graduation she worked as a book cataloger at the University of Pennsylvania, then as an instructor in Washington, D.C., and Boise, Idaho.

At some point after 1904, Ms. Castleman returned to her hometown in Virginia.  Along with her sisters, she formed a library association and eventually established a lending library for Herndon.  She also taught in her mother’s seminary.  During her life, Virginia Castleman published a number of books, including works of fiction, poetry, and children’s books.  She died in Herndon in 1937.  According to the Fairfield (VA) Herald, her funeral was attended by "throngs of the friends of the deceased."