Futures Trading

Futures trading backtesting system

Backtest reporting cycle shortened from 30 days to 2.5 hours

Project background

In order to solve the pain point of the long report generation cycle of the traditional backtesting system, a large domestic futures company introduced LUISUAN Technology GP5000 high-performance all-flash storage to complete the backtesting system POC test. It successfully shortened the generation cycle of transaction analysis reports from 30 days to 2.5 hours, providing efficient data support for trading decisions.

core results

index	Before optimization	After optimization
Transaction analysis report generation cycle	30–45 days	about 2.5 hours
Total backtest calculation time	45 days	About 2.5 hours
Calculate terminal CPU utilization	about 40%	Around 90%
Resource investment	—	Only use 1/3 of the resources to handle 3 times the test tasks

facing problems

Report generation cycle is too long

The insufficient data access performance of traditional storage devices makes backtesting calculation tasks for terabyte-level databases extremely time-consuming, and the report output cycle is as long as 30–45 days, making it impossible to match the rapidly changing transaction needs of the financial market in a timely manner.

Resource utilization is low

The performance of computing terminals and storage devices cannot be coordinated, the computing server CPU utilization is insufficient, network bandwidth is limited, and a large number of hardware resources are in an inefficient operating state, further slowing down the backtesting process.

No room for business expansion

As the volume of transaction data increases year by year, the performance upper limit and expansion capabilities of traditional storage are no longer able to meet the business needs after the test indicators have doubled, and the long-term iterative upgrade of the system faces obstacles.

Three-stage optimization plan

Step One: Infrastructure Adaptation

The production conversion database was directly mounted to the GP5000 high-performance all-flash storage to complete TB-level database data migration; only 1/3 of the device resources were enabled, and the total backtest calculation time was reduced from 45 days to 8 hours and 53 minutes.

Step Two: Architecture and Network Optimization

A 10G dedicated data communication network was built to solve the problem of limited Gigabit network card bandwidth. After optimization, the CPU utilization of the computing server increased to more than 90%, the computing task time was further shortened to 3 hours and 02 minutes, and the storage pressure was only 12%, still retaining sufficient performance margin.

Step 3: Software collaborative tuning

On the basis of fully releasing the hardware performance, the computing software is targeted and optimized to eliminate performance losses at the software level. Finally, the generation time of the transaction analysis report is locked at about 2.5 hours, completing a comprehensive performance upgrade of the backtesting system.

Program inspiration

The futures company's POC test practice proved that the GP5000 high-performance all-flash storage can accurately overcome the data access bottleneck of the financial backtesting system. Through the "storage upgrade to promote architecture optimization + software collaboration" approach, it achieved a leap-forward improvement in backtesting efficiency. Only 1/3 of the resources were enabled to easily handle test tasks that were 3 times the original indicators, and there was still a large amount of performance margin on the storage side, leaving sufficient space for the futures company's future data volume growth and computing node expansion.

In the future, LUISUAN Technology will continue to deepen the storage needs of the financial industry, provide high-performance storage support for core businesses such as transaction systems and risk control systems of more financial institutions, and promote the digital transformation of the financial industry to a higher level.