how to benchmark dbms (sql and nosql) on s390x architecture (IBM mainframes basically)

I have access to an s390x machine, a zbc12 to be precise, with 32GB of RAM, which I can use as a lab for a few months(!).

I wanted to explore this architecture capabilities, especially relating to dbms, and I'd like to test both sql and nosql. comparing that to an x86 architecture, I also have an x86 lying around and I can connect both to the same SAN, so I would be able to properly compare the architectures. how would you guys go about doing such a test, I have little to no experience in benchmarking. what other tests would you like to see? I've got months on this machine and I can play with it as much as I want, any cool and interesting ideas?

Solution 1:

Congrats on access to a Z system.

For the comparison of various databases I can only provide some general guidance. Here are some items to consider as you formulate your plan.

1. Atomicity - divide your databases into ACID versus BASE types as the consistency varies across them and have additional considerations in terms of setup like network, disk, ...

2. System Under Test (SUT) needs to be well defined in terms of the number and types of nodes. Document the underlying network and storage as best you can so people will be able to compare your setup to their intended deployment. What switches are you using? Are you setup for Jumbo Frames? is the storage direct connected or SAN? What are the underlying network infrastructure for both (speed and IOPS).

3. Memory configuration should be well documented in terms of how the DB is configured, make sure that it is consistent, or if you are testing that, document the progression of configurations.

4. If you are comparing ACID v BASE what is the target for consistency and how do you ensure that consistency is complete in terms of replicating / logging the transactions.

5. Consider Recovery Point Objective (RPO) which means how much data am I willing to lose? as well as Recovery Time Objective (RTO) which is when will the DB be available again if there is a failure. This will impact your configuration and assumptions.

6. A consistent client to generate a repeatable load to ensure consistency in tests. Are you scaling the number of clients? Can you do a post validation to make sure that the expected outcome in terms of persistence has been achieved?

There are a number of other factors to consider in doing any test but these will provide a foundation for you to build on.