Elisabeth Batt wrote many children’s books, of which two (as far as I know) were pony books. Some of her books have a Christian background: A Pony to Stay does too.

Finding the books
A Pony to Stay is easy to find in both its incarnations, and not at all expensive. The Wilde Riders is a little more expensive, but is still reasonably priced, and is easy to find.

Links and sources
Many thanks to Hannah Fleetwood for her help with the pictures.