How Propre eases its database management woes

When Propre Japan, a provider of real estate data that helps property buyers keep track of the property market, was using an open source database on Amazon Web Services (AWS) to power its platform, it had to grapple with escalating server costs as usage of its services grew.

Propre’s platform collects records of about 16 million properties each day, detailing each property with about 30,000 features, including its size, surrounding geography, nearby hospitals, as well as information about liveability ratings and various investment indicators.

This cross-border real estate market data is useful for users looking to find the best property from around the world and for property owners to maintain the properties they own efficiently.

In an interview with Computer Weekly, Ryota Negishi, CTO at Propre Japan, said all that data had to be archived regularly, a process that was labour intensive and incurred management costs, which, along with growing server expenses, was unsustainable given the company’s ambition to expand from 17 to 30 countries by the end of 2020.

With that expansion, Propre expects its data volume to triple, spurring it to look for an alternative database that was capable of updating and supporting hundreds of millions of records a day and immediately using the new data for aggregation and analysis.

It eventually settled for Oracle’s autonomous database, taking advantage of its transaction processing capabilities that performed faster than expected – even with heavy data loads.

With the database’s auto-tuning features that automatically optimise memory and disks, Propre’s team could also spend more time planning and developing better user interfaces and additional functionality.

But the migration process was challenging. Negishi said that because the open source database was fundamentally different from the Oracle autonomous database, Propre had to work with an Oracle consulting team to develop a migration plan.

“For example, the data formats in the previous database were different from Oracle’s, so we had to translate one format into another, one at a time,” said Negishi. “And being more familiar with AWS, we had to learn about the Oracle database from scratch, which presented operational challenges.”

Propre, which had to make changes to its applications so that they could work with the Oracle autonomous database, completed the migration in two weeks, with support and training from Oracle. To keep its services running during the migration process, the company also backed up and took snapshots of its previous systems and database, said Negishi.

All that hard work proved worthwhile. Before adopting Oracle’s autonomous database, Propre had to prepare datasets from the big data that it was collecting to deliver services to users, said Negishi. All that heavy lifting is now done in real time.

On the quantifiable outcomes from its use of the autonomous database, Negishi said Propre has already reduced the monthly cost of its servers and database by 25% and expects this figure to reach up to 50% in future.

“At the same time, we’ve improved the performance of the database and we’ve also introduced new services at a much faster pace, with 30% lower maintenance costs,” he said.