This article is written in English and Portuguese
Este artigo está escrito em Inglês e Português
Again, although I have a clear disclaimer on the side, the following does not represent my employer views and in no way is related to any current plans or roadmaps. In fact this is probably the post where I’m closer to the customers than ever.
Why I’m writing this?
I started this article even before v12.10 was out. Due to some constraints I kept the article in draft for a few months. Now is a great time to publish it. The reason for this is that IBM has done an extraordinary thing (not so recently…): It created and made available a platform where customers can make requests for enhancements in some (the list is increasing) of the IBM portfolio products. And guess what? Informix is already there.
I must admit that I have great expectations and I’m pretty enthusiastic about this initiative, for a number of reasons:
- It’s a direct link between customers and IBM. Tell IBM exactly what you need and why… No messing around with sales reps that have other priorities, no PMRs, no user groups filtering
- Being a Web platform it’s easy and very quick to use. It allows insertion, voting, subscribing to updates, commenting and so on… The list of features is fantastic
- Means IBM is willing to take some risks: If a feature is highly requested/voted and seems easy to implement, it will be harder to explain why it’s not implemented (keep in mind that resources are always insufficient…)
- It’s public… again this is a risk for IBM.
All this brings some drawbacks:
- If IBM takes requests seriously (and naturally I believe and hope it will), it may interfere with a more integrated and planned approach to deliver new features
- There will be many requests which are badly explained, or that aren’t clear enough, or that simply can be done in other ways. These will consume resources to filter and may possibly obfuscate the really interesting (note the subjectivity in this sentence) features
- It may expose product limitations to competitors (again I think this a very courageous initiative)
Having explained all this, the link to the site was added to the list on the side of this blog. At the time of this writing there are already 74 requests for Informix. I must admit I’ve added some. The success of this process will depend on customer participation. Review the requests, select the ones you like, vote for them, comment as you think appropriate and create a “watch list” so that you’re notified of changes. And of course, add your own if the features you dream about aren’t’ already there.
I think of this article as a public reflection about functionality I feel is missing currently in Informix. This comes from several customer engagements but are not easy to put into formal feature requests, essentially because those would require a proper business case. So, this is my personal view and feelings, but these are of course influenced by my customer assignments. Meanwhile I’ll try to stick with what seems to me as feasible technical features. I’m leaving out stuff like “I’d like Informix to be the market leader”, “I’d like to see application provider XPTO supporting it” etc.
Naturally, since I don’t work in product development, my thoughts about some of these features feasibility are merely speculation. I’m not able to quantify them in terms of man hours needed to implement and support them. So take all this more like a brain storm. And if you like some of them, vote for them in the site, push for them however you can (IIUG meetings, customer advisory boards, PMRs, through sales representatives, during beta testing programs etc.) Bare in mind that the good people in R&D surely have a lot on their minds, but not necessarily everything we want or need. And of course, since resources are scarce, they must establish priorities. For that they must receive feedback from the field.
For each feature I’ll try to give my idea on how it could be implemented and if this would involve a big development effort (with the limitations stated above). Note that the order presented does not represent a priority order
My personal wish list:
- Have the engine automatically “upgrade” the statistics during/after conversion/upgrade
Allow setting up MACH-11 secondary nodes on heterogeneous platforms
We’ve always had the easiest upgrades in the marketplace. And CONVERSION_GUARD added an unprecedented confidence level to it. But we still require UPDATE STATISTICS to be run after the conversion. Doing it for the procedures is usually very fast. But for all the tables can take a long time (an experienced DBA knows several ways to speed them). I would like to see the engine “upgrade” if needed the existing statistics. This should be done only when the statistics format change (which I believe is rare). If the data collected in the new version doesn’t exist in the old version, than the engine should work without them. The purpose of this would be to be able to open the database for the applications as soon as the conversion ends (typically after a few minutes only).
- How to do it
I really don’t know. To be honest I really don’t know if this is really necessary. How often do the statistics format change? Not many I suppose. The idea of this feature is to make this a supported mechanism that would not require full statistics recalculation. Naturally if the new version gathers more info, the optimizer could possibly miss some info required to achieve the best query plan (but should at least achieve the best ones from previous version)
Reduce the time for doing a supported upgrade into a few minutes. I know…we can do upgrades of MACH-11 clusters with no downtime, but the way we do that is complex (not from the user perspective, but from the engine perspective). And not everybody uses MACH-11
Allow upgrades of secondary servers without having to restore from the primary
Currently we can only have HDR/RSS nodes on the same platform as the primary server. There are good reasons for this and I probably don’t know all of them. But some of these are:
- We need a “bit by bit” image on the secondary nodes. So, platform Endianess becomes an issue
- We need exactly the same layout, so different page sizes becomes an issue
- Even if IBM could fix both issues above, there would still be problems for 3rd party extensions (datablades) than implement their own datatypes and operations
- How to do it
This is the point where my technical ignorance on several implementation aspects raises the risk of saying something really stupid. But I think the first two issues above could be solved. Regarding the endianess, we should take that into account when sending information between the nodes. I believe this is already done for several aspects of the engine (Enterprise replication, distributed queries etc.). With that in mind, I think it would be possible to transform the information, so that although physically different (byte order could change), the “bit by bit image” would be established at a level above, in the Informix low level access mechanisms (reading/writing to disk). Of course this would have to be implemented in every aspect of the engine used to establish and run HDR/RSS nodes (log shipping, backup/restore, ifxclone etc.)
Regarding the page size, this is a limitation, but in reality there are only two default page sizes: 4KB used in AIX and Windows, and 2KB used in all the rest (which are not many – HP-UX, Solaris, Linux….?). Why not let the user define the default page size when creating an instance? Assuming he defines 4KB, it would be acceptable in every platform. So the user would have to consider this before creating the instance.
The last issue, which relates to 3rd party extensions, I’d assume that as a limitation. If the 3rd party implements the needed mechanisms to take the other issues into account, than fine, it would be supported. Otherwise it wouldn’t. My point is that those 3rd party products are not widely used
I think it’s easy to explain… Currently a platform change requires some export/import method, enterprise replication or distributed queries to move the data. With the possibility of having HDR/RSS nodes on different platforms this would be achievable in the easiest manner (there is nothing easier to setup than HDR) and with no down time at all
Currently the only supported way to upgrade a secondary server (HDR or RSS) from version A to B, is to install version B and restore from the primary. In some scenarios, when the servers are close together, this can be achievable in very reasonable time. But for geographically distant servers, this is a real problem. To give you an example we have a customer with a 6TB database… It takes a very large amount of time to take those 6TB from place A to place B. If for a primary server you can do the upgrade in a few minutes, for a secondary server it can be a nightmare
- How to do it
This would be harder than what I’ll explain here, but I believe it would be feasible and it would be a great feature for customers. First, the primary would need to be upgradable without loosing awareness of the secondary servers. Then, assuming we’re upgrading a sever which is a primary server in a MACH-11 cluster, it should be able to save all the new page images (of changed pages during the conversion) in the file system. Very similar to what we do in the CONVERSION_GUARD feature. But instead of the original image, we should store the new image. After this, we should start the secondary with the new version. Once it started it should notice that it’s not running the previous version. As such it should ask the primary if it’s on the same new version. If it is, it should ask if the primary has the “changed pages stored”. If yes, it should accept the transfer of those pages and store them on the local storage. After that it should continue working as before. I can spot a problematic point… This would probably not work if the nodes were not synchronized before the primary conversion started… So some restrictions or mandatory steps could be implemented