Whether a Toad user or not, you can not disregard the beauty of a FREE virtual conference/expo. With budgets slashed and data centers trying to do more with less, this just might be the best bang for your non-buck this year. Not only can you learn the in’s and out’s of Toad for Oracle and DB2 but you will have the opportunity to listen to key database practitioners such as Steven Feuerstein, Guy Harrison, Bert Scalzo, and Tim Fritz to name just a few.
And the best thing, this will be a live event. Sure you can always register and watch on-demand, but wouldn’t you like to touch base with some of these experts and ask a few questions. And you don’t necessarily have to listen to a presentation to do so. The website clearly states that you will be able to connect with the experts in the Expert’s Lounge to chat.
So whether you’re a developer, DBA, architect, or even a C-level, this could be a very relaxed way for you to get some quick and valuable insight from those very close to the Toad product.
I’ve personally chatted with some of the Toad experts before and they are always up front, honest about their product, and will let you know how it can fit within your environment. After all, when you look at these guys and their history, they have truly grown up in the database world, have had to fight many of the fires you’re fighting, and have dedicated themselves to providing solutions that work.
In the recent Heartland Payment Systems security breach, Bank Info Security reports that more than 600 institutions were impacted and had to inform their customers that credit card information had been compromised. The size and impact of this single breach has many institutions (my bank included) sending out letters of potential compromised information. Let’s not forget, honestly I don’t know how Heartland could forget, that identity theft is big business, victimizing 10 million Americans and costing businesses 100’s of billions of dollars a year.
In reading about the Heartland issue, it is very shocking to hear Heartland admit that they were actually PCI compliant—obviously there is something lacking. If you are unfamiliar with what it takes to become PCI compliant, I’d direct you to the Ensuring Your Are PCI Compliant website, also setup by Heartland. Just take a look at the self-assessment questionnaire, form D, as this is what Heartland would have been implementing in regards to protecting credit card information:
Install and maintain a firewall configuration to protect cardholder data
Protect stored cardholder data
Encrypt transmission of cardholder data across open, public networks
Restrict access to cardholder data by business need to know
Restrict physical access to cardholder data
Track and monitor all access to network resources and cardholder data
One would have to question which piece of the puzzle was missing for Heartland. Was it a firewall configuration, securely storing cardholder data, encryption mechanisms, physical access to cardholder data, or the tracking and monitoring of cardholder data? I doubt if we will ever know as Heartland says that they themselves are still piecing together the full story. But I venture to say that their method of implementing a security solution that was PCI compliant lacked adequate cross over between the above mentioned categories.
We can only speculate but the fact that Heartland admittedly was PCI compliant means that there could easily have been some rouge data or maintenance work that left information unprotected and easily accessible. For instance, let’s just say a routine maintenance task to reorganize cardholder left a CSV file out on disk somewhere unprotected. Or maybe some ETL process required some flat file processing of the information. Regardless of the process, many companies are routinely storing sensitive information in flat files that are unprotected.
Flat file formats such as CSV, text, LDIF, and XML that are copied, used in processing, or travel within emails or on laptops put businesses at extreme risk and are a compliance nightmare. Recently I discovered that IRI, maker of CoSort, have a field encryption tool (FieldShield) for these types of flat files. As the name suggests, FieldShield shields and protects fields in flat files with protection like: AES-256 encryption, data masking, lookup pseudonymization, de-ID, or custom functions.
With the high need for compliancy and lack of security concerns within companies, this product has intrigued me so much that I’ll be taking a quick look at it and providing my results in a week or two. Come back and see how my tests went. But until then you might want to give it a try for yourself at IRI free trials.
If you’ve been a DBA for any period of time, hopefully in a shop that practices some form of development cycle, you’ve encountered the troublesome nature of migrating schema changes from development, through to test, QA, and then production. Managing dozens and dozens of development streams with multiple schema transitions can be a mind twister. Even the most diligent of DBA will at one point in his/her career, after making changes within multiple database environments, wonders if the changes were propagated properly.
The big question being: How can I test and verify that everything, or a subset of everything, has been properly propagated from development through to production? Taking Oracle as an example you could start comparing internal views such as DBA_OBJECTS, DBA_TABLES, DBA_INDEXES, DBA_CONSTRAINTS, etc., etc., etc. Or you could do what I did a couple years back, not willing to pay for a tool, and build a series of scripts that used Oracle’s own APIs. Let me be perfectly clear, this can be done.
But can you imagine, building scripts, and maintaining them every time there is a change in the way a particular database vendor maintains database schema information for tables, views, columns, indexes, primary keys, foreign keys, triggers, etc.? Now think about having to test between different databases like from Oracle to MySQL. This can be a nightmare. While I had mastered Oracle, there was no way I was going to master MySQL, DB2, and SQL Server all at the same time and maintain that code.
Do you have multiple development, test, QA, and production systems? Do you worry at night that some piece of your schema migration will fail? Are your eyes tired of checking structures by hand? How about taking a 15 day trial of Database Examiner to compare your databases schemas? Then come back here and report your findings. As a DBA of 20 years, I’d venture to say there are structures in your development, test, QA, and even production that differ and you’re not completely aware of. It’s an easy download, simple setup, uses standard connection streams, and quickly evaluates structures. Give it a try as I’d really like to know your results.
A few months ago I wrote an article titled “SPC Engineer Swallows the Automated Performance Tuning Pill” that told the story of an expert storage engineer who was pitted against the inner workings/features of a storage array; the goal being to optimize performance for a Storage Performance Council (SPC) benchmark. After weeks of tuning and tweaking of unsuccessful performance numbers the engineer reluctantly reverted to an out-of-the-box vanilla system that would tune itself. The end result was a storage system that could out tune the years of built-up knowledge gained by an expert in the storage industry. While we all understand that the knowledge built into the storage device was programmed by experienced professionals, it is the analytical capabilities of the storage system that is able to detect and predict usage patterns as well as configure and distribute data for optimal performance.
So what is Predictive Analytics anyway? Straight from Wikipedia, Predictive Analytics incorporates techniques from both statistics and data mining to analyze current and historical data to make predictions about future events. At the core of predictive analytics is what is called a predictor or variable that when measured can help predict the outcome or behavior for an entity it helps describe. A very simple predictor, for our storage example, might be the speed of a disk drive to determine application response time. Brining multiple predictors together into a predictive model while applying statistical analysis allows us to then forecast the probability of something happening in the future with some level of probability and reliability.
And while a google search on predictive analytics will generate a result set focused mainly on customer retention and market analysis, there is, woven within the search results, a strong push for the use of predictive analytics within many markets and fields of research. A short list includes meteorology (if you can trust the weather), security systems, genetics, economics, collection analysis, fraud detection, insurance, and the list goes on. Somewhere within the list, if you click the right links, you will begin to see predictive analytics being used within the data center to help provide performance intelligence.
But readers beware! While predictive analytics helps drive performance intelligence, we must not forget that Performance Intelligence (PI) is a specialized use case of Business Intelligence (BI) and uses BI technologies to analyze performance-related IT metrics that have been loaded into a data warehouse–allowing users to quickly look at data from several perspectives (slicing, dicing, drill-down, and trend analysis) to see trends or anomalies that might be missed when looking at data in a more granular form. And while BI helps businesses, PI helps IT. Basically PI is BI for IT.
It takes a keen eye to recognize when predictive analytics or performance intelligence is being used out of context to simply gain market traction. It has been my observation that many performance tools have been able to collect and warehouse performance metrics effectively but lack true predictive qualities–relying too heavily on human intervention to provide intelligence. Sure you can click, twist, and drill down through countless of application presentations of detail data but why would you want to? Most of us would end up just like the engineer that had years of experience but had difficulty assimilating all of it into a formal solution.
Lacking in most tools is the ability to change the predictors. So the next time someone asks you to look at their PI, BI, or predictive tool, make sure you ask to change predictors such as workload patterns, storage profiles, system hardware, application configurations, database schema changes, or memory levels. If you can then you may just have a real PI tool. I personally seen BEZVision from BEZ Systems and they actually go beyond just seeing the detail performance statistics for applications and databases and allowed for the changing of predictors. BEZ applies real mathematical models to give IT time and data for making informed and investment savvy decisions regarding the deployment of hardware, software, and personnel for current and future performance.
If you’ve ever needed a load generator to test system loads you might
want to give Swingbench a try. I’ve used this tool on many occasions to
stress disk subsystems, performance tools, and general new features
within Oracle. Setup can be done within your own $HOME directory,
without pestering a system administrator or DBA.
Since I had a need (testing Oracle internals) I thought it would be a good idea to introduce and explain the installation process. Swingbench has different types of benchmarks you can run. For this post I’ve concentrated on the Calling Circle Benchmark (CC).
3. FTP files to your system–placing them in the $HOME directory of the user
4. Uncompress the swingbench file unzip swingbench230391.zip
5. Change to the swingbench directory cd swingbench
6. Install the Jave Runtime Environment that was downloaded from java.sun.com. This was done by placing the self-extracting JRE file in the $HOME/swingbench directory—this allows for portability of a complete swingbench environment, just in case you’d like to move it between systems. Currently that directory is:$HOME/swingbench/jre1.6.0_12 mv ../jre-6u12-linux-i586.bin . ./jre-6u12-linux-i586.bin
7. Edit the swingbench.env file to reflect current Oracle environment. I changed the following two lines: #!/bin/bash # Set the following to reflect the root directory of your Java installation export JAVAHOME=$HOME/swingbench/jre1.6.0_12 # Set the following to the directory where you installed swingbench export SWINGHOME=/home/oracle/swingbench
8. Edit swingbench/bin/ccwizard.xml. A key parameter in this file is the “LightsOut” argument. This allows swingbench to be run in character mode–very helpful to schedule runs through crontab. This file also contains connect information to the Oracle database but is dependent on where and how Oracle has been installed. Most arguments are self explanatory but care should be taken to set the argument values for datafiles of the data and index files to a directory that exists on the database server. I’d also recommend keeping the Mode=InterActive to validate configuration file changes or if you get errors during the process.
Mode of LightsOut will allow you to run from command line, keep as InterActive if you want to use the GUI Increasing the number of accounts will increase the size of the datasets Increasing the transaction count will increase the runtime of the benchmark
8. Create drop, create, and generate ccwizzrd xml files. For simplicity and possibly creating extra scripts to automate the process of running swingbench, the following three files should be created from the ccwizard.xml file edited above.
cd bin cp ccwizard.xml ccwizard.xml-drop cp ccwizard.xml ccwizard.xml-create cp ccwizard.xml ccwizard.xml-generate
You then need to edit one line in each of these three files, such that:
9. Create/Modify ccconfig.xml for running benchmarks. There is a sample ccconfig.xml in the swingbench/bin/sample directory. This file should be copied to the swingbench/bin directory and modified to suite the magnatude of benchmark being run. For instance you might want to change the NumberOfUsers. cp sample/ccconfig.xml .
I changed the following lines in the ccconfig.xml file: <UserName>CC</UserName> <Password>CC</Password> <ConnectString>//ludwig/db11FS</ConnectString>
11. You may want/need to make changes to your Oracle installation. The following commands are suggested areas to alter a default installation of Oracle. The actual directory path to specific files may be different depending on the mount point and Oracle SID.
alter database datafile ‘/oradata/db11FS/system01.dbf’ resize 1024m; alter database tempfile ‘/oradata/db11FS/temp01.dbf’ resize 2048m; alter database datafile ‘/oradata/db11FS/users01.dbf’ resize 500m; alter database datafile ‘/oradata/db11FS/undotbs01.dbf’ resize 2048m;
alter database add logfile ‘/oradata/db11FS/redo04.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo05.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo06.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo07.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo08.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo09.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo10.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo11.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo12.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo13.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo14.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo15.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo16.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo17.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo18.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo19.log’ size 50m; alter database add logfile ‘/oradata/db11FS/redo20.log’ size 50m; alter database drop logfile group 1; alter database drop logfile group 2; alter database drop logfile group 3; alter database add logfile ‘/oradata/db11FS/redo01.log’ size 50m reuse; alter database add logfile ‘/oradata/db11FS/redo02.log’ size 50m reuse; alter database add logfile ‘/oradata/db11FS/redo03.log’ size 50m reuse;
ALTER DATABASE DEFAULT TABLESPACE “USERS”; ALTER SYSTEM SET open_cursors=5000 SCOPE=BOTH; ALTER SYSTEM SET sessions=2000 SCOPE=SPFILE; ALTER SYSTEM SET processes=5000 SCOPE=SPFILE; ALTER SYSTEM SET sga_target=’1536M’ SCOPE=BOTH;
11. Validate sqlnet.ora file on database server. The sqlnet.ora file is in the $ORACLE_HOME/network/admin directory and should have the EZCONNECT entry as shown below. If there is no sqlnet.ora file that is acceptable also.
NAMES.DIRECTORY_PATH= (TNSNAMES, EZCONNECT)
12. Create the tables for the benchmarks. cp ccwizard.xml-create ccwizard.xml ./ccwizard SwingBench Wizard Author : Dominic Giles Version : 2.3.0.391
Running in Lights Out Mode using config file : ccwizard.xml 13. Generate data for a benchmark run cp ccwizard.xml-generate ccwizard.xml ./ccwizard SwingBench Wizard Author : Dominic Giles Version : 2.3.0.391
Running in Lights Out Mode using config file : ccwizard.xml 14. Run a benchmark with minibench. This is a small GUI that gives displays the status of your benchmark run. minibench uses the configuration file swingconfig.xml. Since there are multiple types of benchmarks Swingbench is capable of we need to copy our benchmark to this file. cp ccconfig.xml swingconfig.xml ./minibench
15. Optionally run the character based benchmark tool ./charbench -c ccconfig.xml Author : Dominic Giles Version : 2.3.0.391
Results will be written to results.xml. Hit Return to Start & Terminate Run…
Recently, the Wall Street Journal article titled: “Tech Spending to Decline in ‘09“, Forrester Research states, “Spending on information-technology goods and services this year will contract 3% compared with 2008 to $1.66 trillion.” The economy can change very quickly and with IT budgets falling many companies are trying to orchestrate some form of contingency plans to stay competitive. What many are finding is that during these uncertain times efficiency, innovation, and new ways of thinking are king.
As part of these contingency plans, many companies are turning to open source software and its ability to allow IT to cut costs. Specifically, open source can cut costs by its open availability for free download and no licensing fees. Up for debate is open source’s ability to extend hardware life, be more resilient, reliable, and modifiable, and require less administrative overhead.
The ability for open source to deliver cost saving is, in my opinion directly related to the skill sets within IT. It makes no sense to deploy open source solutions in a production environment without an adequate support system behind it. Many organizations soon find out that deployment into a production environment is much different than just tinkering around or prototyping a system. Code maturity, quality of design, testing practices, and productization (installation, administrative control, APIs, and documentation) are now major concerns for IT management. The questions raised and skill sets required to implement often convince management that open source can only be effectively done with purchasing a supported open source solution or getting assistance from integrators that understand the nature of open source.
So it is no real surprise to me that Black Duck continues to experience rapid growth as reported in a recent press release. As a global provider of products and services Black Duck helps accelerate software development through the managed use of open source software and has experienced a 42% gain in bookings over last year.
I would have to believe that Black Duck success is rooted in their ability to combine open source software with internally developed and third party code. This hybird approach enables companies to chose a best fit scenario. Listening to Kenneth Goldman, executive vice president and chief financial officer, says it all, “We attribute much of our growth and success during such challenging economic times to Black Duck’s core value proposition — giving out customers the solutions they need to stretch resource, remain competitive and prepare for the future”.
This week I was able to get around to a couple of posts on my ITtoolbox blog that took a look at being a DBA when confronted with data integration and open source solutions.
A few of the highlights, if I may where:
IDC reports that the digital universe could grow to a projected 1.8 zettabytes in 2011
While data integration seems straight forward, DBAs are often left to our own vices
Gartner recommends cost-cutting tactics when confronted with data integration problems
We often choose the easy road of downloading and using “free” open source tools for data integration. Let’s just not forget nothing is really free.
There are some good points and bad points about open source
We may need a near- open source solution or at least a product that let’s us try and buy without having to sign extensive service contracts or purchase a monolithic software package/solution
Quite a few years ago I came across a short British film from the
1920’s that has become a German New Year’s tradition. This film has
somewhat of a cult status and I have adopted it to be one of my own for
New Year’s eve.
Take a look at it for yourself, it cracks me up every year. Maybe you’ll make it one of your traditions as well.
I’ll be the first to admit that, while I was at work, watched cats scamper across a treadmill on YouTube today. And I have started to be more active in the number of webinars, video conferences, and news reals I watch. I would say my usage has at least doubled in the last 6 months where I’m watching something at least every three days or so.
I think our infatuation and willingness to watch something online is probably two fold. One, we are simply in front of the computer for so long these days (I spent 73 hours this week) that we just want a quick break now and then to relieve stress or gain some quick information without reading. The second reason is somewhat related in that we are so involved with online content and detached that social networking creates, as it is intended to do, a common bond between online people, thus pulling us deeper and deeper into accepting more and more content.
While I’m sure I haven’t touched on every aspect of what is driving us to become more accepting of online content, Gartner has a current press release in which they state “The popularity of video among consumers will fuel a similar interest in video within enterprises, leading to the severe disruption of existing content strategies“. A couple important take-aways form this press release would be:
Software for managing images and video is the fastest growing segment of the content management market
Video use is growing with about 73% of Internet users watching a video at least once a month
While the take-aways are very positive and, in my opinion, show some exciting times ahead. I’m personally thinking, and making a note, to see how fesible it might be to have some bite-size videos now and then instead of simply writing out a blog post. I wonder if we will get to that point where our blogsphere will begin recording instead of typing. Will our chat truely chat instead of limiting ourselves to bursts of keystrokes?
One can hopefully see the benefit here as communities evolve into constant video conferences. Imagine having multiple video feeds going at the same time and maintaining multiple conversations. Sounds a bit confusing and almost impossible to maintain. But don’t you think that our inginuity can solve that problem. I never thought I could maintain multiple chat sessions going, be typing a blog, and listening to a conferance call all at the same time before. But now I do it with much ease.
My mind is overwhelmed with possibilities here–thinking that there will also have to be some grand infrastructure to keep this new revolution of video going. Imagine the networking issues, the storage issues, the application interfaces, and the list goes on. While the Internet has been around for a while, oh how it has grown, we are, in my opinion, only on the begining edge of realizing how to facilitate and handle informaiton flow.
If one didn’t know better, one might suspect that economic, green IT, and compliance we all conjured up by some form of green-hungry-bureaucratic-money-loving devil. I just think that it interesting that these three challenges facing ITdepartments seem to be coming to a head all at the same time. Granted,compliance has been around a while but there just hasn’t been the same type of enforcement and new regulations coming down from the SEC as in the past.
Gartner finds that power consumption in data centre will increase 1,600% between 2005 and 2025.
In a recent IDC Survey on Green IT, 80% of executives consider it increasingly important for suppliers offer green equipment
SEC announced their Fiscal 2008 Enforcement Results that showed a general uptrend in litigation that it forecasts will extend well into 2009 and beyond.
All this uncertainty, massive layoffs, cut workforces, and the desire to do more with less will surely cause the best of us to wonder how IT will keep up. Well, I stumbled across a new set of webpages put out by Quest Software that will empower companies to pinch a penny and cut costs across theenterprise. Included are briefs, case studies, and video that suppliesinformation on automation, lowering administrative costs, meet compliances, improve performance, etc. Properly titled “Survive and Thrive”. Check it out.
If you’ve ever needed a load generator to test system loads you might
any companies are trying to orchestrate some form of contingency plans to stay competitive. What many are finding is that during these uncertain times efficiency, innovation, and new ways of thinking are king. 
