ProductsDesktop Server For Scientific Computing For IBM POWER For IBM System z For SAP Business Applications Red Hat Network Satellite ManagementExtended Update Support High Availability High Performance Network Load Balancer Resilient Storage Scalable File System Smart Management Extended Lifecycle SupportWeb Server Developer Studio Portfolio Edition JBoss Operations Network FuseSource Integration Products Web Framework Kit Application Platform Data Grid Portal Platform SOA Platform Business Rules Management System (BRMS) Data Services Platform Messaging JBoss Community or JBoss enterprise
SolutionsApplication development Business process management Enterprise application integration Interoperability Operational efficiency Security VirtualizationMigrate to Red Hat Enterprise Linux Systems management Upgrading to Red Hat Enterprise Linux JBoss Enterprise Middleware IBM AIX to Red Hat Enterprise Linux HP-UX to Red Hat Enterprise Linux Solaris to Red Hat Enterprise Linux UNIX to Red Hat Enterprise Linux Start a conversation with Red Hat Migration services
TrainingPopular and new courses JBoss Middleware Administration curriculum Core System Administration curriculum JBoss Middleware Development curriculum Advanced System Administration curriculum Linux Development curriculum Cloud Computing and Virtualization curriculum
ConsultingStandard Operating Environment (SOE) Strategic Migration Planning Service-oriented architecture (SOA) Enterprise Data Solutions Business Process Management
Issue #9 July 2005
- Red Hat GFS now supported with Oracle RAC
- Enterprise data sharing with Red Hat Global File System
- Best practices with Red Hat GFS
- The Linux Logical Volume Manager
- Open source high-availability clustering
- Limiting buffer overflows with ExecShield
- 64-bit computing: Co-existing in a 32-bit world
- Video: Now showing: Red Hat Summit keynotes
- Managing your projects with Planner
- Join Red Hat at Linux conferences in San Francisco and South Wales
- Video: Giving control back to the customer
- Video: Leading French retailer chooses Red Hat for critical sales and customer service system
- FUDCon convenes in Karlsruhe
- Red Hat Enterprise Linux development environment now pre-configured on VMWare
From the Inside
In each Issue
- Editor's blog
- Red Hat speaks
- Ask Shadowman
- Tips & tricks
- Fedora status report
- Magazine archive
Red Hat Global File System now supported with Oracle RAC
by Matthew T. O'Keefe
- The raw Devices approach: Oracle 9i RAC configuration
- A Solution: Red Hat Enterprise Linux, Red Hat GFS, and Oracle 9i RAC
- About the author
Oracle® 9i RAC is a scalable, highly-available, clustered database that runs efficiently on Red Hat® Enterprise Linux® and low-cost servers and storage. Oracle 9i RAC and Oracle 10g clusters allow groups of Red Hat Enterprise Linux servers using shared storage to work together to more quickly process transactions and complete queries, without using expensive UNIX®-based symmetric multiprocessors. It is possible to achieve very high scalability on industry-standard servers with Oracle clustering and Red Hat Enterprise Linux: in December 2003, Red Hat Enterprise Linux and Oracle RAC set the world record for the TPC-C benchmark, achieving over 1 million transactions per minute on a shared storage cluster of 16 industry servers. Red Hat GFS is currently the only third-party cluster file system validated by Oracle for use with Oracle 9i RAC clusters.
To achieve the best performance and scalability using Oracle 9i RAC, it's necessary to have the right software to manage the shared storage hardware environment required by Oracle 9i RAC. The storage management issues with Oracle 9i RAC include the following:
- A separate Oracle installation must be performed on each Oracle 9i RAC node. More than 100,000 files must be installed on each node (this includes the Oracle home and root directories). Home and root directories must be separately managed for each node since these directories cannot be shared.
- Unless Oracle 9i RAC is deployed on top of a file system, a minimum of nine raw devices (one for each table space) must be managed, plus additional raw devices for user space. Raw devices cannot be resized. Raw devices appear as "unused" space and can be inadvertently written over by other applications if proper care is not taken.
The raw Devices approach: Oracle 9i RAC configuration
Let's look further at some of the challenges in deploying Oracle clustering directly on storage devices without a cluster file system. A typical Oracle 9i RAC installation for eight nodes is shown in Figure 1, Oracle 9i RAC cluster using raw devices for storage.
Several tasks must be performed by the system administrator when installing Oracle 9i RAC directly on raw storage devices. These tasks include:
- Creating multiple Oracle instances
An Oracle home directory for each node (and node-specific information for each node) must be created. An eight-node configuration will have:
- Eight local disks, one for each node (or similarly, eight physical or virtual storage devices on a storage network)
- Eight Oracle home directories
- Node-specific information on each of the eight local disks—requiring each local disk to be managed and protected
- A raw storage device must be created and used for each table space
This approach results in a minimum of 10 raw devices to manage, including devices for Oracle tables (system, undo, redo, temp, users, tools, index) and user space data. In Red Hat Enterprise Linux 3, it's important to remember that only 255 raw devices may be created per Red Hat Enterprise Linux server.
- Other servers that may share the same storage device (such as over a SAN) cannot see the raw devices that belong to Oracle 9i RAC. Therefore, Oracle data can be inadvertently overwritten by other servers that share the storage device. In addition, raw devices cannot be backed up using standard, file-based backup approaches.
- Unnecessary downtime and administrative effort
Once created, it's important to remember that raw devices cannot be dynamically expanded. Therefore, the table spaces mapped to these devices cannot be expanded dynamically while the Oracle cluster remains online. Resizing raw partitions is difficult, and a backup version of the table must be written back to the larger partition once it is created offline. In addition, updates to the Oracle home directories on each node are required whenever the Oracle program files are updated.
A Solution: Red Hat Enterprise Linux, Red Hat GFS, and Oracle 9i RAC
Instead of mapping Oracle tables, logs, and archives directly to raw devices, a better approach is to map these structures to files. Since Oracle 9i RAC uses a cluster, its necessary to have a file system that works with shared storage as well. Such a file system is known as a cluster file system (CFS). Red Hat GFS is a CFS that works very well with Oracle clusters. A typical Oracle 9i RAC solution with Red Hat GFS is shown in Figure 2. Oracle 9i RAC cluster using Red Hat GFS for storage.
Oracle 9i RAC is made to access table data, logs, and archives through GFS, which clusters multiple nodes so that all nodes can have equal access to the same files. GFS supports direct I/O access to disks for maximum performance. Configuration and manageability is significantly improved.
The benefits with using the Red Hat Enterprise Linux, GFS, and Oracle 9i RAC include the following:
- A single Oracle instance: Support for a shared Oracle home and centralized node-specific information makes Oracle 9i RAC much easier to install and manage. Over 100,000 Oracle files and executables are installed in a single Oracle home directory. The Oracle home directory is then shared by all Oracle 9i RAC nodes. Node-specific information accessed by Oracle 9i RAC can be centrally stored under Red Hat GFS—eliminating the need to separately manage this information for each node.
- High-performance (direct_io) datafile storage: Create as many data files as are required by the database.
- Downtime to reconfigure and reformat disks for Oracle 9i RAC is eliminated and data integrity is assured.
Using Red Hat GFS to manage the shared storage environment required by Oracle clustering greatly simplifies the task of the database administrator and system administrator to support Oracle 9i RAC clusters. Red Hat Enterprise Linux and GFS make administrators more productive because all administrative tasks can be accomplished on a single cluster node in a consistent way for the entire cluster. Oracle and Red Hat are working closely together to insure that Red Hat infrastructure software, including Red Hat Enterprise Linux and GFS, are the best solution for managing the shared storage environments required by Oracle clustering.
A guide to installing Red Hat GFS with Oracle clusters can be found on redhat.com.
For more information on Red Hat GFS, refer to the following articles: