Information service patterns, Part 1: Data federation pattern

Document options Print this page E-mail this page Hey there! developerWorks is using Twitter Follow us Rate this page Help us improve this content Level: Intermediate Dr. Guenter Sauter (gsauter@us.ibm.com), Senior IT Architect and Manager, IBM Corporation Bill Mathews (bmathews@us.ibm.com), Senior IT Architect, IBM Corporation Mei Selvage (meis@us.ibm.com), Software Engineer, IBM Corporation Dr. Eoin Lane (eoinlane@us.ibm.com), Senior Software Engineer, IBM Corporation 28 Jul 2006 The data federation pattern virtualizes data from multiple disparate information sources. The pattern creates an integrated view into distributed information without creating data redundancy while federating both structured and unstructured information. This article describes the federation of structured information (data) with a focus on the SOA context. This pattern specification helps data and application architects make informed decisions on data architecture and document decision guidelines. Show developerWorks content related to my search: data federation Introduction Many organizations struggle with the disparity and distribution of information. In many cases, users spend a large amount of time searching for and manually aggregating, correlating and correcting relevant information instead of acting on the insight that they gain from the information. This widely recognized challenge also occurs when implementing a Service-Oriented Architecture (SOA). Often, core services require aggregated, quality information from multiple diverse sources. Several concepts and technologies address those integration needs. Data federation is one of them. Data federation aims to efficiently join data from multiple heterogeneous sources, leaving the data in place -- without creating data redundancy. The data federation pattern supports data operations against an integrated and transient (virtual) view where the real data is stored in multiple diverse sources. The source data remains under the control of the source systems and is pulled on demand for federated access. This article highlights the value of the data federation approach. After describing the context in which we apply data federation, we discuss the problem that this pattern addresses, as well as the solution. We characterize the applicability of this pattern based on non-functional requirements (see the Considerations section). Some known usages of this pattern illustrate our experience in applying this pattern. We conclude by summarizing the focus areas, risk areas and constraints of this pattern. Back to top Value proposition of the data federation approach Transparency of underlying heterogeneity With data federation, the consumer will see a single uniform interface. Location transparency means the consuming application of the pattern does not need to be aware of where the data is stored. Nor does it need to know what language or programming interface is supported by the source database, thanks to invocation transparency. For example, if SQL is used, it does not matter to the application what dialect of SQL the source supports. The application also does not need to know how the data is physically stored due to physical data independence, fragmentation and replication transparency -- or what networking protocols are used, known as network transparency. Time-to-market advantage An application that is a consumer of the data federation server can interface with a single virtual data source. Without using the federation pattern, the application must interact with multiple sources individually through different interfaces and different protocols. Studies have shown that using the data federation pattern helps to reduce development time significantly when multiple sources have to be integrated. See Resourcessection for more information. Reduced development and maintenance costs Many consumers may potentially need the same -- or very similar -- integrated information. In one approach, each consumer has its own implementation for aggregating information from diverse sources. Alternatively, the integrated view is developed once, and it is leveraged multiple times and maintained in a single place, thus creating a single point of change. This approach reduces development and maintenance costs. Performance advantage An implementation of the data federation pattern with a specific focus on advanced data processing technology has, in many cases, proven to have superior performance characteristics compared with a home-grown approach to aggregate information (see the Resources section for more information). By leveraging advanced query processing capabilities, the federation server can optimally distribute the workload among the federation server itself and the various sources. It will determine which part of the workload is most effectively executed by which server in order to optimize response time. Reusability advantage After applying the data federation pattern to a particular integration scenario, the result of this specific federated access can be provided as a service to multiple service consumers. For example, an integration scenario may require retrieving structured and unstructured insurance claim data from a wide range of sources. In this example, the data federation pattern can provide the solution to integrated claims data which is then surfaced through a portal to a claims agent. The same federated access can then be leveraged as a service to other consumers such as automated processes for standard claims applications, or client facing web applications, for example. Improved governance Governance is a key underpinning to the SOA lifecycle. The governance process is enhanced by the use of patterns by reinforcing best practices with predictable outcomes. Reuse of proven flexible patterns in the development and creation of systems can both ensure consistency and quality and reduce maintenance costs by having a single source to update with changes. Back to top Context Mergers and acquisitions among companies and organizations often require data and application architects to integrate disparate data sources into a unified view of the data. Consumers of this integrated information are traditional applications that interact directly with databases and require access to an extended set of data sources. The decision on how best to provide this unified view are often set against the availability of tooling, experience, expertise and culture of the organization. Using traditional legacy architectures, the time, effort and cost associated with the integration may exceed the business benefit. A pattern-based information services approach, when implemented within a services based environment, can enhance the reusability characteristics of the system over time. Information services are part of the core backbone of a SOA. These information services provide Create-Read-Update-Delete (CRUD) access to domain information. They also surface information processing capabilities such as the results of analytical and scoring algorithms, data cleansing rules, etc. For the purposes of this article, we will focus on information integration services that provide a unifying view of the data, which often involves the integration of a bewildering array of disparate backend sources, and services. When applying the data federation pattern, we need to distinguish between two contexts: the traditional, non-SOA context, addressed by many previous applications, and the SOA context which is the focus of this article. It is important to keep in mind that SOA is an architectural approach which results in reusable services that in many cases extend the capabilities of existing non-SOA implementations. Traditional context In what we refer to as the traditional context, a reporting application in a bank might need to analyze credit card transactions. Considering the volume of this data -- there are many million of transactions per day -- it is not efficient to store all this information in the analysis warehouse. Much older data is very infrequently accessed, as is certain context information, such as a flight itinerary. Storing all credit card transaction data -- current and outdated, core and related -- in the warehouse negatively impacts the performance. A better solution is to separate the two types of data: frequently used, more recent credit card transactions are stored in a warehouse while older information is stored on tapes, for example. However, the reporting application should not need to be aware of this data distribution which can be provided through the federated approach. Figure 1. Traditional data federation pattern   In this traditional context, applications typically use standard relational interfaces and protocols to interact with the federation server, SQL and JDBC/ODBC for example. The federation server in turn connects through various adaptors, or wrappers, to a variety of data sources such as relational databases, XML documents, packaged applications and content management and collaboration systems. The federation server is a virtual database with all of the capabilities of a relational database. The requesting application or user can perform any query requests within the scope of their access permissions. Upon completion of the query a result set is returned containing all of the records that met the selection criteria. This is illustrated in Figure 1. The figure is intended to illustrate that the traditional implementation may be based upon a relational application programming interface (API) using SQL (JDBC/ODBC) or XQuery. SOA context In an SOA context, a service getCustomerCreditCardData might need to retrieve comprehensive information about a customer and his recent credit card transactions. This information might not reside in a single system. Customer information might be stored in a customer master data management system, or multiple repositories, and credit card transactions might be stored in another data source. Data federation joins the information from multiple sources so that it can be surfaced as a service to the consumer. In this SOA context, the federation server can act as a service provider and/or a service consumer which leverages SOA conforming interfaces. Note that this does not preclude the server from also providing support for the traditional, relational interfaces. The breadth of support is an implementation decision which is beyond the scope of this discussion. When the data federation server exposes integrated information as a service provider, a service consumer can access the integrated information through a service interface such as WSDL and HTTP/SOAP or other agreed-to bindings. The data federation server can consume -- in order to integrate -- services provided by multiple information sources. The thought behind using the data federation pattern in the SOA context is to leverage and reuse integrated information, that is, information integration services in an extensible manner for a variety of consumers. The modeling and definition of services is a key aspect of SOA. It is a commonly acknowledged best practice to design services so that they provide reuse and/or cross-enterprise interoperability and/or business process enablement of information or functionality. Many if not most successful SOA projects focus first on the most important, most widely used business functions that are exposed as services. Due to the key role that those services play, they often span multiple backend systems. Gathering information from multiple heterogeneous sources is therefore an important requirement and capability that SOA relies on. The service is not a query as in the traditional data access context, rather, it is a request for a business entity (or entities) which may be fulfilled by the federation service through a series of queries and other services. Figure 2. Data federation pattern in an SOA context   Enabling information integration services within SOA requires additional functionality that encapsulates a federated access within a service-oriented interface. This is accomplished through Information Service Enablement. The purpose of this component is to surface certain federated queries in a service-oriented interface. For example, a federated query might be written in SQL and might specify access to product information. Through the Information Service Enablement component, this federated query can then be surfaced as a service, for example, defined by SCA or WSDL. The service that implements access to product data can then be shared across and beyond the enterprise. Solutions that apply the data federation pattern in the traditional context leverage the advantage of the declarative and flexible nature of SQL. With appropriate security credentials, consumers can access any data in the source through an almost unlimited number of different SQL queries. Consumers have great flexibility in what to access and the format in which the result is returned. Although this flexibility is a great advantage in many situations, it also increases the complexity for consumers. Consumers have to understand the source data model and how to construct the result from this underlying source model. The larger the source data model, the more complex this task can become. An SOA approach focuses first on defining and sharing a relatively limited number of the most critical business functions as services within and across the enterprise. Therefore, service-oriented interfaces are much more focused on the limited number of specific information requests that need to be surfaced. Developers benefit from this clear and narrow focus since they need less time to design the information request. They can simply select the appropriate service out of a relatively limited number of options. Back to top Problem statement In today's information-driven environment it is very common for architects and developers to implement a data federation solution. The challenges they face are usually affected by a number of architectural decisions, which may be driven by constraints that are technical, business or contractual in nature. This scenario includes several of these common constraints. First, data necessary to support the information access requirements of the project resides in multiple sources and must be integrated and provided as a single result to the consumer. Next, the target data sources cannot be replicated or copied in order to fulfill the access requirement. Lastly, the solution must integrate within an existing SOA while still supporting the traditional non-SOA applications as depicted in Figure 3. Figure 3. Heterogeneous interface access   Back to top Solution goals As described in the problem statement, it is the goal of this approach to avoid data redundancy when providing an integrated view over heterogeneous sources. The data federation server -- that is, the component that implements the data federation pattern -- must provide standard query interfaces for the non-traditional SOA context. This ensures that a wide range of traditional database applications can consume the federated data. The federation server must also provide query optimization capabilities in order to respond to the request most efficiently. The distribution and heterogeneity of data in this context requires a strong emphasis on how to best translate access to the integrated view and how to decompose and distribute the workload. When supporting write access to this integrated view, the federation server must synchronize the manipulation of data in the various sources into a logical unit of work. This ensures that the atomicity, consistency, isolation, and durability (ACID) criteria for transactions are met and that referential integrity is enforced. In addition to these goals that address this traditional context, the approach must fit within a SOA. This will allow a wide range of consumers throughout and beyond the enterprise to effectively reuse the integrated view(s). Potential consumers of a federated access in a SOA are applications, portals and activities within a business process that need access to distributed information. For example, a manufacturer might define a service that retrieves real-time inventory information from heterogeneous sources. Internal applications as well as external business partners then access the same service, leveraging a consistent and most efficient implementation of this federated access. Back to top Solution description In both the traditional as well as the SOA context, the data federation server provides a solution to effectively join and process information from heterogeneous sources. This pattern realizes a synchronous, real-time integration approach to distributed data. The data federation server is responsible for receiving a query directed at an integrated view of diverse sources. It transforms it using complex optimizing algorithms that result in breaking the query down into a series of sub operations referred to as query partitioning and rewrite, applying the sub operations against the appropriate sources, gathering the results from each source, assembling the integrated results and finally returning the integrated results to the origin of the query. This processing sequence is done synchronously and in real time. Design time characteristics The data federation pattern requires the mapping of data elements from various data sources that are within the scope of the integrated view. For example, customer information, such as name and address from a policy holder, as in the example mentioned above, might be stored in a single table in one database and in multiple tables in another database. In order to build an integrated view, those different types of representations need to be mapped to the common view. The mapping can be performed manually by human actors or assisted by state-of-the-art tools based on various mapping algorithms which also capture any necessary transformation requirements. This allows the data federation server to receive queries against the integrated view and to calculate the optimum number and types of sub operations to perform. When applying the data federation pattern in an SOA context, a set of federated queries need to be enabled and registered as services within SOA. For example, the integrated view to retrieve critical structured and unstructured information about a policy holder, for example name, address, status, claim documents, repair estimates, and risk rating can be enabled as a service and shared among multiple consumers. The result of mapping in design time are typically federated views, similar to relational database views, which then can be deployed or created on the federation server. Run time The data federation server receives a request to the integrated view. According to the mapping definition, the federation server breaks down the federated query into multiple sub operations. Multiple factors influence this step: Where does the data reside that is necessary to respond to the federated query? What operations are necessary to transform the heterogeneous representations of the sources such as different data types, normalized model vs. non-normalized, for example, into the common integrated view? The federation server uses the mapping information to address those questions. There are a number of other factors that influence the federated query processing which require information beyond the mapping specification such as: What operations are supported by the systems managing data sources and which have to be compensated by the federation server? What are the performance implications when executing a set of operations in the sources vs. the federation server? Which operations should the federation server delegate to the sources in order to better exploit their capabilities, to reduce data transfer, and to optimize the overall performance? The answer to these questions requires knowledge of the source system and its query processing capabilities. In order to address the latter question, the federation server must also utilize a range of information about the operational environment as well as statistics of the source databases. Once the federation server has determined the best execution strategy of all sub operations, it connects to the data sources -- both structured and unstructured information -- in order to retrieve relevant data, potentially using source-specific interfaces. According to the overall query execution plan, the sub operations are then applied at the sources. The result is received and aggregated into the result of the integrated view. The result is then returned to the consumer. In the SOA context, the consumer submits a request via a predefined request format to the federation server. The federation server transforms the request into the corresponding SQL queries, or view definitions, to support the service. From there on, the same query decomposition, optimization and execution steps are performed as described above. The only difference in the SOA context is in the final step. The federation server translates the result of the traditional data federation approach into a service response and then returns it to the service consumer through the predefined service interface. Figure 4. Sequence diagram for data federation   The functionality of the data federation pattern can be implemented using either database-related technologies such as optimizer or compensation, or by home-grown applications. Due to the complexity of query optimization over heterogeneous sources, it is an industry best practice to use a data federation implementation that leverages query optimization technology as provided by most database management systems. Back to top Considerations When applying the data federation pattern, it is important to understand its characteristics and how it is affected by the non-functional requirements described below. It is important to note that the non-functional requirements we have outlined do not take cache and data replication patterns into consideration. It is our belief that when adopting patterns that one starts with the basic patterns -- Data Federation in this example -- which can then be extended with additional patterns that address the additional non-functional requirements and functionality needed for the service. Cache and data replication patterns can be used to supplement the data federation or in the creation of a composite pattern. These patterns, and any other pattern that might be used in the overall implementation should be used cautiously as they may hinder the fulfillment of some non-functional requirements for which data federation has been chosen in the first place. For instance, they may increase data latency and create data redundancy. One needs to understand the trade-off points based on non-functional requirements and architectural decisions. All characteristics of the non-functional requirements apply to both the traditional non-SOA context as well as to the SOA context. They include: Data security Only users and applications which have the appropriate credentials in the integrated sources are allowed to access the integrated view. This may be further restricted. One of the main reasons to apply this pattern is to leverage existing source systems with their data and capabilities. As a consequence, architects often intend to also leverage existing security mechanisms such as authentication and authorization of the source systems. Due to the heterogeneous and distributed nature of this environment, some challenges regarding single sign on and global access control might arise which are outside of the scope of the data federation pattern. In order to address those challenges, architects will need to combine the data federation pattern with other security-related patterns. Data latency The data federation pattern allows for real-time, integrated access to sources with the highest level of data currency. Source data volatility Due to the real-time access to source data upon receiving a request to the integrated view, data federation will always return the most current source information. Since the data federation pattern does not create copies of source data, source changes do not have to be propagated or processed in this approach. Data consistency and quality With the increase in frequency that complex data cleansing, standardization and transformation operations need to be performed, the probability of a negative impact on the overall response time increases. This is due to the real-time, synchronous nature of responding to requests in the data federation pattern. Any additional transformation will mean additional workload when responding to an integrated query. It is a best practice to minimize the complexity and number of field transformations required. Data availability The availability of integrated data depends on the availability of the data federation server and the integrated source servers at the time of the request. If one of the servers or any connection between the federation and the source server fails, the integrated view is not available. Impact of model changes on integrated model A very significant benefit to the data federation pattern is the ability to mask off many model changes which may be implemented in the source systems. The ability to accommodate the changes within the federation server can reduce the probability of exposing these changes to the initiator or consumer of the service. Further, changes can be made in the integrated view without requiring any changes to be propagated to the models for the data sources. Frequency of transaction execution A request to a federated server is executed synchronously. As soon as the response is received, the requester can invoke a subsequent request. The federated server should support concurrent requests initiated by multiple requesters. Highly frequent subsequent requests should have the same performance characteristics as a single request. An exception may occur if a source -- or a connector between the federation server and the source -- has specific characteristics that cause response performance degradation when frequently accessed. The ability of the federation server to execute transactions at a high rate is determined by the rate at which the federation server can access the source systems and the ability of those source systems to respond. Transaction concurrence In many cases, the data federation server has very similar characteristics than a database or content server. The ability to efficiently manage concurrent access is determined by the performance characteristic of the data federation server as well as the integrated source servers. Performance and transaction response time The transaction response time is determined by many factors, including: Complexity of the federated query: how many sub-operations such as filtering, joining, sorting, and so on, does the federation server need to execute to perform the query Query optimization and processing capabilities of the data federation server: how sophisticated is the design of federation server to take a federated query, break it down in sub-operations and optimize it, for example, applying certain sub-operation first such as the filter to reduce the data set and then performing other sub-operations such as sorting Data volume: the higher the data volume, the longer each sub-operation and therefore also the complete query will take network bandwidth: the throughput of a network connection between the federation server and a source impacts how quickly the federation server can access the source and therefore also the overall response time of the federated query CPU utilization: differences in resource utilization of the machines that the federation server and the data sources run on need to influence which sub-operations of the overall federated query are performed at the federation server vs. at the sources, if possible Query processing capabilities at the source servers: some data source servers have specific characteristics and limitations on how they process and optimize queries that impacts the overall performance The ability of the federated server to identify the optimal query strategy for each data source: if the federation server is aware of query processing capabilities of the source servers, it can determine what type of sub-operations to delegate and what sub-operations to perform at the federation server layer The response time of a query against a virtual database, implemented by the data federation pattern -- fetching data from distributed sources -- might be slower than the same query against a single physical database with the same capabilities. The difference in response time will vary depending on the factors listed above. As a consequence, alternative patterns that provide the integrated data set in a single physical database can allow for improved response times. Some implementations of the data federation pattern are capable of sending some or all of the sub-operations (sub-queries) in parallel to the integrated source systems. The parallel processing of sub-operations can significantly improve the response time. Create-read-update-delete (CRUD) profile Most data federation implementations support a various degree of read and write access. Some implementations coordinate a logical unit of work for write operations, known as a two-phase commit. In most cases, the data federation pattern is used for read access because of the complexity of write access. Without two phase commit support, the requester is responsible to ensure consistency among the sources when updating data. Because two phase commit generally requires a transaction manager, the degree of support for write access may vary depending upon the implementation of the transaction manager in addition to the functional capabilities of the source server with respect to applying and committing changes. Data volume per transaction The response time is influenced by the volume of data that need to be moved from remote source to federated server per transaction: the higher the data volume, the slower the response time. It is critical for the federation server to optimize the federated query so that the minimal amount of data has to be transferred between the federated server and the sources, especially when federated data volume is large. It is also important to understand the capacities and bandwidth supported by the network infrastructure and the impact that may have on the volume and frequency of data transferred. Solution delivery time As described in the value statement, data federation can greatly improve delivery time when integrating various sources. Skill set and experience The data federation pattern focuses on the integration of data sources and provides a single system image through a data-oriented interface. When surfacing integrated information as services, developers will also need to understand SOA concepts, standards and technologies. Reusability Logic on defining data access and aggregation can be reusable across different projects. Cost of maintaining multiple data sources Data federation does not reduce the cost of maintaining multiple data sources but greater benefits can be achieved due to integration and reuse of existing data sources. Cost of development It is relatively cheap if utilizing the best-of-breed federation engines, assuming a federated server infrastructure is in place. Type of target models This article has focused on federation for structured data. Today, the most common model is the relational model with the SQL standard. XML and XQuery are emerging standards with an increasing adoption in information management. Implementations of the data federation pattern typically support at least one of those models, sometimes both. Most implementations of the data federation pattern have a relatively strong focus on one -- or a very limited number of -- target models in order to process requests most efficiently. Assured delivery and logical unit of work In the IBM SOA Reference Architecture an enterprise service bus (ESB) is a key component of the infrastructure. One of the responsibilities of the ESB is to provide assured delivery. Due to the complexity of coordinating a logical unit of work, such as through two phase commit protocol in a federated environment, not all implementations of the data federation pattern support this functionality. When using federation servers that support this functionality, they need to be carefully analyzed on their database locking strategies to avoid negative impact to the performance of source systems. Resource utilization The federation server only utilizes resources when it processes a request that it receives from the consumer. The level of utilization on the federation server is also determined by the complexity of the request: the more complex the request, the more complex the task of finding the optimal plan how to decompose this federated request into sub operations. Another factor to the resource utilization is the percentage of sub operations that need to be executed in the federation server, for example to compensate lacking functionality in the source systems, vs. sub operations that can be pushed down to the source systems. Also, the amount of data that is received from the source systems and needs to go through the federation server impacts the resource utilization. Transformation capabilities The focus of the federation pattern is to leave data in place and to provide a real-time, virtual, integrated view. The solution approach in this pattern does not have any limitations on what transformations can be applied. Basic transformations are used in many implementations in order to convert heterogeneous source formats into the common view at the federation layer. However, complex transformations have a negative impact on the performance of the federation pattern and make this pattern less applicable for those scenarios. Therefore, most implementations of the data federation pattern focus less on complex transformation capabilities and more on query optimization technologies. Type of source model, interfaces, protocols Data federation addresses the problem of integrating data from heterogeneous source models and includes concepts to map those different source models into the common model at the federated layer. Implementations of the data federation pattern vary in their capabilities which specific source models they can integrate. Scope and size of source models The size of source models, number and type of attributes, may negatively impact the mapping task during runtime when mapping the underlying sources to the integrated view. The broader the scope,for example, the larger the number of attributes to be accessed, the longer it may take to identify corresponding elements. Impact of federation server workload (transaction volume) to sources The federation server forward for each request that it receives sub-operations to source systems. This impacts the resource utilization at the source systems negatively in that they need to respond to the sub-operations from the federation server. The more requests the federation server receives, the more sub operations will be sent to the integrated sources. Back to top Conclusion We described the data federation pattern as an approach to data operations against an integrated and transient (virtual) view where the real data is stored in multiple diverse sources. We focused primarily on the SOA context within this article. We will conclude by summarizing when to apply and when not to apply the data federation pattern and to list important constraints. Focus areas to apply the data federation pattern When time-to-market is one of top development priorities, data federation offers access to information sources quickly without lengthy information management infrastructure changes. Data federation supports requirements with respect to the replicating and duplication of data by enabling access to the data as it resides in the source. These requirements can be in response to regulations or rules restrict the movement or replication of data, e.g. subscription data or the commingling of personal information from different countries. Real-time access to distributed information as if from a single source. Information can be both structured and unstructured data. Flexible and extensible information integration approach for a dynamically changing environment, in particular schema evolution: due to the lack of data redundancy, changes in the federated schema reduce the impact of changes to integrated systems. The advantage of data federation is best exploited when a modest number of requests are received against limited result sets sized from multiple consistent, complementary data sources. Risk areas to apply the data federation pattern Integration scenarios that require complex transformations to build the integrated view will have a negative impact on the response time particularly in this approach. Source servers may be negatively impacted by increased workload when they have to return data that is requested in a federated query. In order to process a request to the integrated view, the federation server will send sub operations to integrated sources. The more complex those sub operations are and the more frequently they are sent to the sources, the more additional workload the source servers need to manage. Scenarios that result in large intermediate result sets being moved from the target data sources to the federation server may have significant performance implications. Situations in which applications require a relatively high degree of availability of the integrated data may not be good candidates to apply this pattern. The availability of the integrated data is wholly dependent upon the availability of all federated and source servers involved in the process as well as availability, capacity and responsiveness of the network. Constraints when applying data federation pattern Many implementations of the data federation pattern have limited capabilities to manipulate data. Many use SQL as the programming language and can only support SQL transformations. The performance largely depends on the sophistication of vendor specific implementation in terms of caching capability, understanding heterogeneous data sources and formulating optimal federation quires and execution path. Read-write access to different information sources - in particular when coordinating a logical unit of work - is constrained by the vendor-specific support. Back to top Product mapping The following IBM products implement this pattern: IBM® WebSphere® Information Integrator Standard Edition, Advanced Edition, Advanced Edition Unlimited allows applications to federate data among sources on distributed platforms (LINUX / UNIX / WINDOWS). Consumers can access through a SQL interface federated information from a wide range of data sources such as Informix, Cloudscape, Oracle, SQL Server, Microsoft Excel, XML files, etc. This product can also be combined with the following two products to aggregate structured, unstructured data and the valuable assets from the mainframe platforms. WebSphere Information Integrator Classic Federation provides a SQL interface through a wide range or data sources on the mainframe, such as DB2®, IMS, VSAM, IDMS, Adabas, etc. One of the key capabilities of this product is to map mainframe data structures to the relation model without coding so that developers with very limited mainframe knowledge can flexibly and efficiently access mainframe data. WebSphere Information Services Director surfaces information management capabilities as services. It packages information integration logic, cleansing rules, information access etc. as services. This insulates the developer from the underlying provider of this functionality. Most relevant to this article is its capability to surface a federated access through a service oriented interface such as EJB, JMS, or Web services. This product provides the foundation infrastructure including load balancing and fault tolerance for Information Services. It realizes the Information Service Enablement component that is illustrated in Figure 2. WebSphere Information Integrator Content Edition provides a uniform interface over federated content management systems. Typical content access operations are provided over a wide variety of content sources; among these is IBM DB2™ Content Manager. Back to top Acknowledgements We would like to thank Jonathan Adams, Kyle Brown, Lou Thomason, and Fan Lu for their support in writing this article and in developing this pattern. Resources Extending Portals with DB2 Information Integrator (developerWorks, February, 2004) shows you why and how to extend the reach of pre-built components and make it easier to develop custom components.  IBM Federated Database Technology(developerWorks, March 2002) describes the technology that enables clients of IBM's federated database engine to access and integrate the data and specialized computational capabilities of a wide range of relational and non relational data sources.  SOA and Web services: New to SOA and Web Services  Patterns: Information Aggregation and Data Integration with DB2 Information Integrator is an IBM Redbook that documents and prototypes the role of DB2 Information Integrator technologies and architecture in IBM's Patterns for e-business using a typical customer insight e-business scenario.  Performance of DB2 Information Integrator in a J2EE Environment with Multiple Data Sources (developerWorks, June 2003) compares performance results for queries run with federation vs. queries using direct access.  Using DB2 Information Integrator for J2EE Development: A Cost / Benefit Analysis, (developerWorks, May 2003) describes using DB2 Information Integrator to access disparate back end data sources, including Oracle, DB2 UDB, and Microsoft Excel. About the authors Dr. Guenter Sauter, senior IT architect and manager, leads the team that is working on information service patterns which address the linkage between information management and SOA. He is also the demo architect for information management, demonstrating capabilities across the complete IBM Information Management portfolio. Bill Mathews is a senior IT architect in the IBM Financial Services Sector for the Americas and is the architectural lead for Information Integration. He has over 25 years of experience in the IT industry, is an Open Group Master Certified IT Architect and holds IBM IT Architect and Consultant certifications. His areas of expertise are information integration, enterprise application integration, and Web application development. Bill holds a Bachelors of Science degree in Computer Science from Hofstra University and a Masters of Business Administration degree from Union College. Mei Selvage is a SOA data architect with extensive hands-on experience in various information management areas and Service-Oriented Architecture (SOA). Her mission is to bridge the gap between SOA and information management. Her research interests include: information management and integration patterns (both structured and unstructured data), data modeling, metadata, faceted search, human collaboration and SOA. Dr. Eoin Lane, senior solution engineer, is the lead for harvesting and developing of application pattern from key IBM SOA engagements and driving those patterns through IBM pattern governance process to accelerate adoption. Eoin also specializes in Model Driven Development (MDD), asset based development and Reusable Asset Specification (RAS) to facilitate SOA development.

China grants more patents in 2008

China grants more patents in 2008

(Xinhua)
Updated: 2009-01-30 16:59

BEIJING -- China granted 412,000 patent rights in 2008, up 17.1 percent over the previous year, patent authorities reported on Friday.

These included more than 352,000 domestic applications and 60,000 from abroad, according to the State Intellectual Property Office (SIPO).

The office also revealed it received over 828,000 patent applications last year, an increase of 19.4 percent year-on-year.

A spokesman for the SIPO said the office did not compile statistics on the number of applications rejected.

Yin Xintian, another spokesman with SIPO, said, "Patent grants in high-tech fields, such as audio-visual, optical and semiconductor technologies, still far behind foreign countries".

Each application took one to three years to be processed, depending on their complexity.

China has granted 2.5 million patent rights since 1997.

Stem Cell Research: The Quest Resumes

Thursday, Jan. 29, 2009
Stem Cell Research: The Quest Resumes
By ALICE PARK

Scientific inspiration can come from anywhere — a person, an event, even an experiment gone awry. But perhaps nothing can drive innovation more powerfully than the passion born of tragedy. Or, in Douglas Melton's case, near tragedy. The co-director of the Harvard Stem Cell Institute (HSCI) is one of the leading figures in the search for cures for presently incurable diseases, and his breakthrough work is challenging many long-held beliefs about the ways biology and human development work.

But it was a very personal experience that brought Melton to stem cells, one that 17 years later he still finds difficult to discuss. When his son Sam was 6 months old, he became ill with what his parents thought was a cold. He woke up with projectile vomiting and before long began taking short, shallow breaths. After several hours, he started to turn gray, and Melton and his wife Gail brought the baby to the emergency room. For the rest of that afternoon, doctors performed test after test, trying to figure out what was wrong. "It was a horrific day," says Melton. (See the top 10 medical breakthroughs of 2008.)

It was not until that evening that a nurse thought to dip a testing strip into Sam's urine and they finally got a diagnosis. The boy's body was flooded with sugar; he had Type 1 diabetes. Then, as now, the disease had no cure, and patients like Sam need to perform for themselves the duties their pancreas cannot — keeping track of how much glucose they consume and relying on an insulin pump to break down the sugars when their levels climb too high. The diagnosis changed not only Sam's life but the lives of his parents and older sister Emma as well. Throughout Sam's childhood, Gail would wake every few hours during the night to check his blood sugar and feed him sugar if his concentration fell too low or give him insulin if it was too high. "I thought, This is no way to live," says Melton. "I decided I was not just going to sit around. I decided I was going to do something."

Trained as a molecular biologist in amphibian development, Melton began the work he pursues today: trying to find a way to make insulin-producing cells by using stem cells. "It was a courageous thing to do because he was at the pinnacle of his career," says Gail. "He brought home textbooks on the pancreas to figure it all out." Nearly two decades later, Melton is convinced that stem cells will be a critical part of new therapies that will treat and maybe cure not only diabetes but also other diseases for which there are no answers today.

Melton's confidence is testament to the extraordinary advances in stem-cell science, some of which have brought the promise of breakthrough therapies for conditions like diabetes, Parkinson's and heart disease closer than ever before. The cells filling petri dishes in freezers and incubators in Melton's lab and others around the world are so vastly different — in provenance, programming and potential — from the stem cells of just two years ago that even the scientists leading this biological revolution marvel at the pace at which they are learning, and in some cases relearning, rules of development. Until recently, the field has revolved around either embryonic stem cells — a remarkably plastic class of cells extracted from an embryo that could turn into any of the body's 200 tissue types — or their more restricted adult cousins, cells taken from mature organs or skin that were limited to becoming only specific types of tissue. On Jan. 23, after nearly a decade of preparation, the Food and Drug Administration approved the first trial of an embryonic- stem-cell therapy for a handful of patients paralyzed by spinal-cord injuries.

But today the field encompasses far more than just embryonic and adult stem cells; it has expanded into the broader field of regenerative medicine, and Melton's lab at Harvard is at the vanguard, bringing the newest type of stem cells, which do not rely on embryos at all, closer to the clinic, where patients will actually benefit. Last summer, Melton stunned the scientific community with yet another twist, finding a way to generate new populations of cells by reprogramming one type of fully mature cell so it simply became another, bypassing stem cells altogether. "If I were in high school, I can't imagine anything more interesting than stem cells," says Melton. "This is so cool. It's so amazing that cells in the body have this potential that we can now unlock by asking question after question."

A Battle Joined
That hidden power in each of us did not become obvious until 1963, when Canadian researchers Ernest McCulloch and James Till first proved the existence of stem cells, in the blood. These cells possess the ability to divide and create progeny — some of which will eventually expire, others that are self-renewing. The pair irradiated mice, destroying their immune cells. They then injected versatile bone-marrow cells into the animals' spleens and were surprised to see a ball of cells grow from each injection site. Each mass turned out to have emerged from a single stem cell, which in turn generated new blood cells.

That discovery led, 35 years later, to James Thomson's isolation of the first human embryonic stem cells, at the University of Wisconsin in 1998. And that milestone in turn inspired researchers to think about directing these cellular blank slates to eventually replace cells that had been damaged or were depleted by disease. The key lay in finding just the right recipe of growth factors and nutrients to induce a stem cell to become a heart cell, a neuron, an insulin-making cell or something else. It would take decades, the researchers all knew, but new therapies were sure to come.

Then, in 2001, everything changed. The use of discarded embryos made embryonic-stem-cell research deeply controversial in the U.S. Citing moral concerns, then President Bush restricted federal funding for the study of human embryonic stem cells. Under the new policy, U.S. government funds could be used only to study the dozens of embryonic cell lines already in existence — many of which proved not to be viable.

Read Stem Cells: The Hope and The Hype.

See the Year in Health, from A to Z.

The decision sent some leading scientists abroad, to Britain, Singapore and China, where the governments were more receptive to their work. Others who stayed behind but lacked private funding shifted their attention from embryos to the less versatile adult stem cells. Federally backed scientists, like Melton, who continued embryonic work were forced to adopt a byzantine system of labeling and cataloging their cell cultures and equipment so that government money was not used to grow forbidden cells — and government microscopes were not even used to look at them.

Those days may soon be over. Barack Obama campaigned on a promise to lift the research ban and support "responsible oversight" of the stem-cell field. For scientists, that means "we can stop the silliness," says Melton.

As welcome as that change will be, it may be less urgent now — owing primarily to the work of scientists like Melton. While embryonic stem cells remain the gold standard for any treatments that find their way into the clinic, newer techniques using the next-generation stem cells may soon surpass the older ones.

The Fighter
In looks and demeanor, Melton is the quintessential professor, soft-spoken and thoughtful, someone who appears more mentor than maverick. Born and raised on the South Side of Chicago, he developed an early fascination with animal development; that curiosity led to a bachelor's degree in biology at the University of Illinois in 1975, then a second undergraduate degree, in the history and philosophy of science, at Cambridge University on a Marshall Scholarship. Melton remained there for his Ph.D. work, studying under Sir John Gurdon — the first to clone a frog. At Harvard, Melton teaches a frequently oversubscribed undergraduate course on science and ethics, in which he uses his keen sense of logic to provoke. When the class discussed the morality of embryonic-stem-cell research, Melton invited Richard Doerflinger of the U.S. Conference of Catholic Bishops to present arguments against the field. Melton asked Doerflinger if he considered a day-old embryo and a 6-year-old to be moral equivalents; when Doerflinger responded yes, Melton countered by asking why society accepts the freezing of embryos but not the freezing of 6-year-olds.

Clearly, Melton does not shrink from a fight. As Washington's squeeze on stem-cell research tightened in the early part of this decade, he decided to take action, providing life support for what remained of the U.S. stem-cell community. Not convinced that an entire field could make much progress relying on a few dozen cell lines of questionable quality, in 2004 he used funds HSCI receives from the Juvenile Diabetes Research Foundation and the Howard Hughes Medical Institute, as well as from Harvard alumni, and developed a more streamlined method for generating stem-cell lines from embryos. He created more than 70 new ones and has since distributed 3,000 copies to scientists around the country for free.

"Doug drew a line in the sand," says Alan Trounson, president of the California Institute of Regenerative Medicine, the organization charged with dispensing state money for embryonic-stem-cell research. "He turned the tables on an Administration that was incredibly negative toward stem cells and showed [it] we are not going to tolerate being put out of this field by ideological views that we don't think are correct." Melton's motivation was, again, both professional and intensely personal. Two months after Bush announced his ban, Melton's daughter Emma, then 14, also received a diagnosis of Type 1 diabetes.

In part owing to the restrictive U.S. policy, the momentum in stem-cell research seemed to shift overseas. In 2004, South Korean researcher Hwang Woo Suk announced that he had generated the first human embryonic stem cells from healthy people — and in the following year, from afflicted patients themselves — using an abbreviated cloning method. The latter feat would mean that cardiac patients could essentially donate themselves a healthy new heart without fear of rejection.

The news was huge — but it was also a lie. In 2006, Hwang admitted he had falsified his results. (Melton's colleague at HSCI, Kevin Eggan, finally created embryonic stem cells from patients in 2008.) Although Hwang became a pariah, he had the right idea. Melton and others had been trying to do just what the Korean scientist claimed to have done — grow a new population of a patient's own cells. The key to the process is a supply of fresh, good-quality human eggs, which incubate skin cells taken from a patient. Building up such a stockpile, however, proved practically impossible. The egg-extraction process is invasive and carries certain risks; after the state of Massachusetts prevented donors from being compensated for their eggs, out of fear the women would feel coerced, HSCI ended up with only one volunteer out of more than two years of recruiting.

Melton faced mounting political pressure too. In 2004, voters in California approved a measure providing $3 billion in state funding to embryonic-stem-cell research. That threatened to draw scientists in the stem-cell community west, and Melton took pains to foster a "band of brothers" mentality. "I tried to create a cocoon here," he says, "and tell people that your job is to focus on the science. Don't worry what the politicians say." By then, Melton's team was one of only a handful in the country working on embryonic stem cells and was making headway in teasing apart the myriad critical steps needed to guide these impressionable cells into becoming insulin-generating cells. Both as a scientist and as a father, Melton remained convinced that the federal restrictions simply could not survive. He continued to insist that "the science is so significant that it will change the policy."

And then, astonishingly, it did. In June 2006 a modest researcher from Japan made a startling announcement at the International Society for Stem Cell Research conference in Toronto. Shinya Yamanaka quietly described a study in which he took skin cells from a mouse and stirred them in with varying genetic cocktails made from a recipe list of 30 genes known to be important in development. When he hit on the right four genes and inserted them into the cells aboard retroviruses, he wiped the cells clean, reprogramming them and returning them to an embryo-like state without ever creating the embryo. Four genes, he told his audience, was all it took to undo a lifetime's worth of delicate genetic tapestry. No need for eggs, no need for embryos. Could it be that easy? Were the debate and controversy over embryonic stem cells now rendered moot? "It was unquestionably unexpected," says Melton of the breakthrough.

Read a TIME cover story on Stem Cells.

See the Year in Health, from A to Z.

A year later, Yamanaka followed up his work by reporting success with the same four factors in turning back the clock on human skin cells. At about the same time, in Wisconsin, Thomson achieved the same feat using a different cocktail of genes. With those studies, what became known as induced pluripotent stem cells (iPS cells) were suddenly a reality. Never mind the frustratingly fickle process needed to create embryonic stem cells; this was something any molecular-biology graduate student could do. "We figured somebody would have success with reprogramming. We just thought that somebody would come along a generation from now," says Dr. David Scadden, Melton's co-director at HSCI. "Yamanaka threw a grenade at all of that, and now all of the doors are open."

Beyond Stem Cells
Melton, for one, isn't wasting any time before running through those doors. The iPS technology is the ultimate manufacturing process for cells; it is now possible for researchers to churn out unlimited quantities of a patient's stem cells, which can then be turned into any of the cells that the body might need to repair or replace.

Before that can happen, however, Melton wants to learn more about how diseases develop. And iPS cells make that possible too. For the very first time, he can watch Type 1 diabetes unfold in a petri dish as a patient's cells develop from their embryonic state into mature pancreatic cells. The same will be true for other diseases as well. "There is a good reason we don't have treatments for diseases like Parkinson's," says Melton. "That's because the only way science can study them is to wait until a patient appears in the office with symptoms. The cause could be long gone by then, and you're just seeing the end stages." No longer. Now the major steps in the disease process will be exposed, with each one a potential target for new drugs to treat what goes wrong. "This is a sea change in our thinking about developmental biology," says Dr. Arnold Kriegstein, director of the Institute for Regeneration Medicine at the University of California, San Francisco. "I consider it a real transformative moment in medicine."

The true power of reprogramming, however, does not stop with the stem cell. This summer, Melton flirted with the rules of biology once again when he generated another batch of history-making cells, switching one type of adult pancreatic cell, which does not produce insulin, to a type that does — without using stem cells at all. Why, he thought, do we need to erase a mature cell's entire genetic memory? If it's possible to reprogram cells back to the embryo, wouldn't it be more efficient in some cases to go back only part of the way and simply give them an extreme makeover? Using mouse cells, Melton did just that, creating the insulin-producing pancreatic cells known as islets. "The idea now is that you can view all cells, not just stem cells, as a potential therapeutic opportunity," says Scadden. "Every cell can be your source."

Realizing that potential — and with it, the prospect of successful treatments for conditions like Parkinson's or diabetes — may still be a few years away. Even iPS cells have yet to prove that they are a safe and suitable substitute for the diseased cells they might eventually replace in a patient. Ensuring their safety would require doing away with dangerous genes that can also cause cancer, as well as the retroviral carriers that Yamanaka originally used. Melton's team has already replaced two of the genes with chemicals, and he anticipates that the remaining ones will be swapped out in a few years. There are also hints that the iPS cells' short-circuited development makes them different in some ways from their embryonic counterparts. In mice, embryonic stem cells can generate a new mouse clone; iPS cells from the animals have so far stopped short of the same feat, aborting in midgestation, suggesting that some development cues may be missing. "It certainly makes me cautious," says Eggan.

Even if iPS cells do not prove as stable and as versatile as embryonic stem cells when they're transplanted into patients, they remain a powerful research tool. And if nothing else, they will have opened our eyes to the remarkable plasticity of biology and made possible new ways of thinking about repairing and replacing damaged tissues so we may consider not only treating but also curing disease. "It's a wonderful time," says Scadden. "Keep your seat belt on, because this ride is going to be wild."

For patients like Sam and Emma Melton, that ride carries with it the possibility of being free of the insulin pumps and injections they endure to keep their blood sugar under control. "I definitely think about how my life would be different if there is a cure," says Sam. His father is keenly aware that the ability of stem cells and reprogramming science to provide that cure is far from guaranteed. But his initial confidence in the power of the technology hasn't waned. "Everything we learned about stem cells tells us this was a really powerful approach," he says. "It would be a great shame if we let it wither and just go away." Melton, for one, is determined not to let that happen.

Science in Steps

A decade of conflicts and breakthroughs

1998
James Thomson, U of Wisconsin, isolates human embryonic stem cells

2001
President Bush restricts federal funding for research on human embryonic stem cells

2004
Douglas Melton of Harvard creates more than 70 embryonic-stem-cell lines using private funding and distributes free copies of the cells to researchers around the world

2006
Shinya Yamanaka, Kyoto University, turns back the clock on mouse skin cells to create the first induced pluripotent stem (iPS) cells, or stem cells made without the use of embryos. He uses only four genes, which are inserted into a skin cell's genome using retrovirus vectors

2007
Yamanaka and Thomson separately create the first human iPS cells

2008
July
Kevin Eggan at Harvard generates the first patient-specific cells from iPS cells — motor neurons from two elderly women with ALS

August
Melton bypasses stem cells altogether and transforms a type of mouse pancreatic cell that does not produce insulin into one that does

September
Konrad Hochedlinger at Harvard creates iPS cells in mice using the common-cold virus rather than retrovirus vectors — an important step in making the technology safer for human use

October
Melton's team makes human iPS cells by replacing two of the four genes, known to cause cancer, with chemicals. All four must be swapped out before iPS-generated cells can be transplanted into people

October
Yamanaka creates mouse iPS cells using safer plasmids of DNA instead of retrovirus vectors

Read Stem Cells: The Hope and The Hype.

Peter Klaus und Reinhardt Jünemann

Peter Klaus und Reinhardt Jünemann

Die beiden neuen Mitglieder der Logistik Hall of Fame: Professor Peter Klaus (li.) und Professor Reinhardt Jünemann (re.) (Foto: LOGISTIK inside / Scheutzow)
Logistik Hall of Fame: Logistik-Vermesser und Materialfluss-Papst ziehen in die Ruhmeshalle ein

München. Professor Peter Klaus von der Universität Erlangen-Nürnberg und Reinhardt Jünemann, der ehemalige Institutsleiter und Gründer des Dortmunder Fraunhofer-Instituts für Materialfluss und Logistik, sind am Freitag in die Logistik Hall of Fame aufgenommen worden. „Mit ihren Leistungen haben die beiden außergewöhnlichen Wissenschaftler die Logistik in Deutschland maßgeblich vorangebracht“, begründet Anita Würmser, Jury-Vorsitzende und Chefredakteurin von LOGISTIK inside die Entscheidung der unabhängigen Expertenjury.

Professor Peter Klaus (* 1944) ist bekannt durch die von ihm ermittelten zwei wesentlichen Kennzahlen der Logistik. Nach seinen Berechnungen hat Logistik in Deutschland ein Marktvolumen von 205 Milliarden Euro und beschäftigt 2,7 Millionen Menschen. Durch diese Werte hat die Logistik ihr Image als Verkehrslärm und Staus produzierendes Ungetüm verloren. Logistik wurde populär und gilt in Politik und Wirtschaft seitdem als Boombranche und Jobmotor. Sie wird jetzt als eigene Branche beziehungsweise eigener Industriesektor anerkannt.

Professor Reinhardt Jünemann (* 1936) zählt zu den Vätern der industriellen Logistik und hat das Fraunhofer-Institut für Materialfluss und Logistik (IML) in Dortmund gegründet. Das Institut gehört heute weltweit zu den führenden Forschungseinrichtungen. Auf Jünemanns Pionierleistungen gehen wichtige Impulse für automatische Materialflusssysteme, die Simulationstechnik sowie für die Informations- und Kommunikationstechnik in der Logistik zurück.

Seit dem Jahr 2004 wurden bisher elf verdiente Logistiker in die Logistik Hall of Fame aufgenommen: Helmut Baumgarten, Eugene Bradley Clark, Heinz Fiege, Huge Fiege, Klaus-Michael Kühne, Malcom McLean, Taiichi Ohno, Hans-Christian Pfohl, Hanspeter Stabenau, William H. Tunner und Horst Wildemann.

Die Logistik Hall of Fame ist eine Initiative des Fach- und Wirtschaftsmagazins LOGISTIK inside aus dem Münchner Verlag Heinrich Vogel. In die Ruhmeshalle der Logistik werden Persönlichkeiten aufgenommen, die sich durch ihre Leistungen außerordentlich um die Entwicklung der Logistik im deutschsprachigen Raum verdient gemacht haben. Die Logistik Hall of Fame ist jederzeit kostenlos im Internet unter http://www.logistik-hall-of-fame.de zugänglich und bietet eine Reihe von Informationen und Bilder über die logistischen Meilensteine und deren Macher. (sv/pi)

Swinging the axe

Jan 29th 2009 | DAVOS
From The Economist print edition

Job-cutting has begun in earnest. But will the axe be wielded wisely?

Illustration by Claudio Munoz

THE headlines screamed that January 26th was “Black Monday” for jobs, after firms such as Caterpillar, Corus, Home Depot, ING, Pfizer and Sprint Nextel announced cuts of several thousand jobs each, due mostly to the rapidly deteriorating global economy. Alas, the consensus among the corporate bigwigs gathered this week at the World Economic Forum in Davos was that this marked only the beginning of the axe-swinging, and that there are blacker days to come.

This proved to be one of the big points of difference between the company bosses and the politicians brainstorming in the mountains. The politicians are primarily concerned with restoring demand enough to reverse the rising trend in unemployment; for many of the corporate leaders, ensuring the survival of their firms takes precedence over saving jobs. The difficult decision they face is not whether to cut, but how to do so in a way that strengthens their competitive position in the medium term rather than seriously damaging it.

The gloomy mood among bosses in Davos makes the worst-case scenario outlined in a new forecast from the International Labour Organisation (ILO) seem the most plausible of its possible outcomes. This supposes that if every economy in the developed world performs as it did in its worst year for unemployment since 1991, and every other economy performs half as badly as in its worst year, then the global jobless rate will rise to 7.1% this year—some 230m people, up from 179m in 2007.

The ILO’s most optimistic prediction is that global unemployment will rise only to 6.1% (from 6% in 2008). But that assumes that the world economy performs as the IMF forecast in November: global GDP growth of 2.2% in 2009, with a slight recession in the developed economies. The IMF has since become much glummer: this week it forecast growth of just 0.5%.

Already, firms are starting to find that their first round of cuts after the onset of the crisis is not enough. Caterpillar’s latest cut of 5,000 jobs is in addition to 15,000 already announced. Such is the frenzy of cutting that Challenger, Gray & Christmas, a recruitment firm that tracks employment trends in America, sought a crumb of comfort in its finding that over 50% of firms have cut jobs: it proclaimed in its latest report that “nearly half of employers avoid lay-offs.” But it pointed out that things would be even worse without the various innovative schemes adopted by companies to reduce labour costs without shedding jobs. These include salary cuts, reduced hours and “forced vacations”.

As Challenger suggests, this seems in keeping with the suggestion by Barack Obama in his inauguration speech that people should “cut their hours [rather] than see a friend lose a job.” Already, by way of example, White House staff earning $100,000 or more have had their salaries frozen. Companies including Avis, Starbucks and Yahoo! have announced pay freezes for 2009.

Yet these creative job-saving schemes are unlikely to go anywhere near as far as Mr Obama would like. They may appeal as a way to buy some time as companies try to get a clearer picture of where the economy is heading, or to retain talented workers who are likely to be needed in the future, if not now. But they have little appeal once a firm has decided that it needs to scale back its operations. As the boss of a big American retailer put it privately at Davos, “We have to decide who we want on the bus and to motivate them as much as possible.” Clever ways to share the pain can demotivate everyone, especially if they are seen as merely postponing the inevitable job cuts, making everyone fearful.

Painful choices

Equally candidly, many bosses admit that the crisis is giving them a chance to restructure their firms in ways that they should have done before, but found a hard sell when things were going well. As a rule of thumb, a careful cull of the 10% of lowest performers can make a firm leaner by removing fat without damaging muscle. It is going beyond the 10%, as many firms are now starting to do, that poses the real risks to a firm’s competitiveness.

During the relatively modest downturn at the start of this decade, for example, many professional-services firms cut too deeply, especially in their lower ranks, and found they were poorly positioned when strong growth resumed sooner than expected, says Heidi Gardner of Harvard Business School. Firms built on pyramid structures in which senior managers mentored larger numbers of employees below them suddenly found that, in a growing economy, they lacked the mentors needed to manage the army of new recruits. Instead, they had to re-hire ex-staffers at higher salaries and, in some cases, abandon proven policies of hiring senior managers only from within, says Ms Gardner, who worked for McKinsey at the time.

This crisis is revealing how few firms have really thought through their talent strategies, says Mark Spelman of Accenture. Claims that “our workers are our most valuable assets” are too often platitudes, the emptiness of which is now being revealed. But those firms that have thought seriously about their talent needs have the opportunity to get ahead of those that haven’t, says Mr Spelman, not just by shedding poor performers but also hiring scarce talent from outside, in what is now a buyer’s market. Other tips from Mr Spelman include avoiding voluntary redundancy programmes, which encourage the most employable people to quit, and not firing the newest recruits on a crude “last in, first out” basis, as this cuts off the supply of future talent. Instead, firms should identify which workers they need to keep, and do what they must to retain them.

Governments can play a useful role or a harmful one, depending upon their attitude to companies, says David Arkless of Manpower, an employment-services firm. If they focus on working with firms to smooth the movement of labour to where the future work will be, for example by providing skills training and financial incentives to workers in transition, then the economic downturn could be less painful than now seems likely. (A quick recovery in lending to small businesses, the main drivers of job creation in most countries, would also help.) But if governments try to prevent firms from making the changes to their workforces that they want, the result is likely to be prolonged gloom.

Although Mr Obama’s support for strengthening the ability of unions to enter workplaces is arguably a worrying sign, the American economy is far more accommodating of flexibility in employment than many European countries. Mr Arkless, for one, says that without a dramatic change of attitudes to job-cutting in Europe, “there is no doubt that American firms will come out of this downturn better than anywhere else in the world, due to their flexible employment model.” This will provide no comfort to anyone facing the prospect of unemployment, but it is a message that politicians would do well to take to heart.

Friday, January 30, 2009

Microsoft Searches for Group Advantage

A new search algorithm looks for connections between people.

By Robert Lemos --- technologyreview.com

As part of its efforts to better compete with Google, Microsoft is plumbing the connections between searchers and their contacts to produce better results.

Microsoft researchers are exploring whether using data from several members of a social group--a technique that the company calls "groupization"--can improve search results. Their initial findings, based on experiments involving around 100 participating Microsoft employees, suggest that tapping into different types of groups could produce significantly better search results.

Credit: Technology Review

The team has developed an algorithm that, on average, pinpoints at least one search result for all members of a group that they judge to be better than the results returned using conventional algorithms. The results will be presented at the Web Search and Data Mining Conference in Barcelona in mid-February.

The Microsoft team believes that the approach could help the company overcome an industry-wide plateau in the quality of search results. "Today, search engines are really challenged and are sort of at the cusp of having to know individuals better," says Jaime Teevan, a computer scientist at Microsoft Research and lead author of the paper. "This [research] has the opportunity to enrich that."

The new research is part of Microsoft's efforts to erode Google's massive lead in search. Google currently attracts 63 percent of all searches, according to a 2008 survey by consumer-analysis firm Nielsen, far outpacing both Yahoo's 17 percent share and Microsoft's 10 percent share. Last year, Microsoft attempted to increase its share by acquiring Yahoo, but its initial advances were rejected. Yahoo later wanted to return to the bargaining table, but for the time being, Microsoft is focusing on increasing its audience by enhancing its own search offering.

With an eye on refining search results, Teevan and her colleagues--Meredith Morris and Steve Bush--looked at the way that people with similar interests or attributes search for information. The researchers grouped people using explicit factors, such as their age, gender, participation in certain mailing lists, and job function. In some cases, implicit groups--such as people who appeared to be conducting the same task or appeared to have the same interest--were inferred. The researchers acknowledged that gathering such data in the real world could be tricky. But it could perhaps be collected through registration, by caching previous searches or by tapping into social-networking software.

The Microsoft team found that groups defined by demographics such as age and location have little in common for most searches. However, groups of people with similar interests tend to rank similar search terms highly. The researchers also found that, while people believe that they phrase their queries in similar ways, the idiosyncrasies of search terms vary tremendously.

When asked to identify the pros and cons of telecommuting, for example, one searcher searched for "telecommuting," while others queried "working at home cost benefit" and "economic comparison telecommuting versus office." Knowing that these people have a shared interest could mean better results, Teevan says. "I don't talk about things the same way that you talk about things," she says. "And by using those different ways, [Microsoft is] more likely to find a page where someone talks about something in their own way."

Even if tapping into social groups improves search results, Microsoft will have to significantly improve its search service or introduce major new features to win over Google's loyal followers, says Andrew Frank, research vice president with business-intelligence firm Gartner's media group. "I think that the search category has been so successful for Google, and their dominance is so extreme, that it is hard to imagine a specific tactic that could be a silver bullet to change the trajectory of things," he says. "It will take a lot of effort and a lot of different things to change the overall picture of search."

Efforts to compare the quality of search results from Google, Microsoft, and Yahoo have found that about half of people still prefer Google--a smaller number than Google's actual market share. The difference is the attraction of Google's brand, Frank says. "You have to kind of change the game with search," he says. "It is almost impossible to get people to switch on a large scale just on a feature-function comparison."

In 2008, Google kicked off an experiment in which it allowed users to change the look of their search results, mapping them or placing them on a timeline. Early this year, the company added ability function that lets users reorder their search results through a service called SearchWiki. Yahoo has expanded its research-and-development efforts to try to match its rival's efforts.

Microsoft's Teevan believes, however, that there's still plenty of room for improvement. "Search is a really huge activity on the Web, but right now, we only have a single search tool--the search box--and a list of results," she says. "Groups can teach us a lot."

Thursday, January 29, 2009

The Army's Remote-Controlled Beetle

The insect's flight path can be wirelessly controlled via a neural implant.

By Emily Singer --- technologyreview.vom

A giant flower beetle with implanted electrodes and a radio receiver on its back can be wirelessly controlled, according to research presented this week. Scientists at the University of California developed a tiny rig that receives control signals from a nearby computer. Electrical signals delivered via the electrodes command the insect to take off, turn left or right, or hover in midflight. The research, funded by the Defense Advanced Research Projects Agency (DARPA), could one day be used for surveillance purposes or for search-and-rescue missions.

Cyborg beetle: Shown here is a giant flower beetle carrying a microprocessor, radio receiver, and microbattery and implanted with several electrodes. To control the insect’s flight, scientists wirelessly deliver signals to the payload, which sends electrical signals through the electrode to the brain and flight muscles. Credit: Michel Maharbiz

Multimedia Watch controlled flights of the beetle.

Beetles and other flying insects are masters of flight control, integrating sensory feedback from the visual system and other senses to navigate and maintain stable flight, all the while using little energy. Rather than trying to re-create these systems from scratch, Michel Maharbiz and his colleagues aim to take advantage of the beetle's natural abilities by melding insect and machine. His group has previously created cyborg beetles, including ones that have been implanted with electronic components as pupae. But the current research, presented at the IEEE MEMS in Italy, is the first demonstration of a wireless beetle system.

The beetle's payload consists of an off-the-shelf microprocessor, a radio receiver, and a battery attached to a custom-printed circuit board, along with six electrodes implanted into the animals' optic lobes and flight muscles. Flight commands are wirelessly sent to the beetle via a radio-frequency transmitter that's controlled by a nearby laptop. Oscillating electrical pulses delivered to the beetle's optic lobes trigger takeoff, while a single short pulse ceases flight. Signals sent to the left or right basilar flight muscles make the animal turn right or left, respectively.

Most previous research in controlling insect flight has focused on moths. But beetles have certain advantages. The giant flower beetle's size--it ranges in weight from four to ten grams and is four to eight centimeters long--means that it can carry relatively heavy payloads. To be used for search-and-rescue missions, for example, the insect would need to carry a small camera and heat sensor.

In addition, the beetle's flight can be controlled relatively simply. A single signal sent to the wing muscles triggers the action, and the beetle takes care of the rest. "That allows the normal function to control the flapping of the wings," says Jay Keasling, who was not involved in the beetle research but who collaborates with Maharbiz. Minimal signaling conserves the battery, extending the life of the implant. Moths, on the other hand, require a stream of electrical signals in order to keep flying.

The research has been driven in large part by advances in the microelectronics industry, with miniaturization of microprocessors and batteries.

Thursday, January 29, 2009

Predicting Breakdowns

A new system that monitors the health of vehicles could save money and lives.

By Brittany Sauser --- technologyreview.com

Most new vehicles are studded with hundreds of sensors that collect raw performance data. While beneficial, such information can only be interpreted by the manufacturer or dealer and is usually only read after the car breaks down. Now researchers at Rochester Institute of Technology (RIT) and Lockheed Martin, a security company in Bethesda, MD, have developed a monitoring system that can better assess the health of a vehicle and can alert drivers to any potential problems.

Performance evaluation: Researchers at Rochester Institute of Technology (RIT) and Lockheed Martin have developed a system that monitors and assesses the health of a vehicle and predicts its future performance. The vehicle, such as the military light-armored vehicle, is embedded with sensors that send information to a computer at a central operational center. Shown here are Dave Keegan (right), an RIT student, and Bob Kosty, a technician, looking at the software that analyzes the data to determine how the equipment is functioning and warn of any potential problems.
Credit: Laura W. Nelson, RIT

The system uses a network of embedded smart sensors that are strategically located near automotive components that are prone to problems. The information is sent wirelessly to a central command center, where it is automatically analyzed by software. The monitoring system is similar to OnStar, an in-vehicle security, communications, and diagnostics system built by GM. But Nabil Nasr, assistant provost and director of the Center for Integrated Manufacturing at RIT, says that the system goes "far beyond" anything commercially available by predicting "future health or failures."

The project is part of a $150 million contract between Lockheed Martin and the U.S. Marine Corp, which is equipping up to 12,000 military vehicles with the new technology. The system can assess the health of military vehicles before they are sent on missions so that commanders can know if a vehicle is up to the task. "It could save money and lives, and extend the lifetime of equipment," says Nasr.

The technology has also been tested in a public-transit bus at the Rochester Genesee Regional (RGR) Transportation Authority for the past 18 months. Eight months ago, a spinoff company called LIBAN formed to develop the technology for commercial fleet vehicles.

The system uses standard sensors--such as temperature, vibration, and electronic sensors, as well as customized smart sensors--to monitor a vehicle. The sensors are placed near different components on the vehicle, such as the transmission, alternator, and drivetrain. "Most systems on the market today are just reporting fault codes coming out of the engine-control module. We are looking at data from individual components to get better details . . . and to predict future conditions," says David Chauncey, CEO of LIBAN.

The data from the sensors is processed by an onboard computer system that analyzes the information. That data is sent at regular intervals to a control center via a cellular network, satellite, or private data network, depending on the customer. "Every vehicle is an intelligent, potential source of information, and we have the technology to make the data useful; we just need to develop communication protocols and standards so we can build the infrastructure to share information, beyond just the manufacturers and dealers," says Kirk Steudle, director of the Michigan Department of Transportation (MDOT), which just announced a partnership with Michigan International Speedway to create an open testing environment for cross-brand vehicle communication.

The "heart of the system" is the data-monitoring software, says Nasr. The RIT researchers have created sophisticated programs for mining, trending, and analyzing the data from the sensors. "The algorithms are extremely valuable because they help us build a model of predictive and condition-based maintenance, so we can predict failures before they occur, and we can make determinations about service based on the actual conditions of the equipment," says Randy Weaver, advanced-technology-systems program manager at RGR. The fare box on a bus, for example, is a key piece of hardware that the system monitors. "It's the number-one reason we change a bus off," says Weaver. If the box becomes jammed, transit personnel cannot accept fares from magnetic-pass holders (smart cards), so it is imperative that transit authorities know in advance the condition of the box.

"The [technology] is critical because our primary service is to keep buses on the road for the community, so if we can prevent a road breakdown during service hours, then the technology pays for itself," says Weaver. RGR plans to integrate the technology into its fleets in the next six months.

Steve Underwood, director for the Connected Vehicle Proving Center at the Center for Automotive Research, in Ann Arbor, MI, says that the technology is really important, especially for reducing costs, and that there is a lot that can be done in addition to measuring the performance of a vehicle. Sensors can be used for incident detection, as well as for identifying traffic patterns and pavement conditions.

While the technology is currently being placed in military light-armored vehicles, LIBAN hopes to also use it in commercial vehicles, such as trucks owned by the U.S. Department of Defense, private-fleet operators, freight haulers, and other public-transit systems.

Thursday, January 29, 2009

Digging Deeper in Web Search

A personalization search tool reveals links buried deep within page results.

By Kate Greene --- technologyreview.com

One of the hottest frontiers in Web search is finding ways to improve results based on the searchers' preferences. Already, when you log in to Google, the search engine tries to personalize results by mining your search history: for an eighth grader who has performed lots of searches on marine life, a search for "dolphins" might provide more results for the animal than for the football team.

Credit: Technology Review

Now Surf Canyon, a startup based in Oakland, CA, is adding its own spin on personalization. Its software, which can be downloaded and installed into Firefox and Internet Explorer Web browsers, enhances individual searches on major search engines by evaluating which links you click on, and then instantly giving you revised search returns--including three sites that relate in some way to the site you clicked on. "We have invented real-time personalization," says Mark Cramer, CEO of the company.

For example, a Google search for "thermoelectric cooler" using Firefox with Surf Canyon installed provides 10 standard results. In my case, the eighth result, from freescale.com, a chip maker, seemed promising. I clicked on it, scanned the page, and then hit the "back" button. When I subsequently looked at the results page, three new suggestions appeared directly under the freescale.com result. Surf Canyon had elevated these links from the earlier 100 pages of results because its algorithm determined that these recommendations related to the information on freescale.com, including technical explanations of how thermoelectric coolers work.

Crucially, these new results are cleverly slipped into the search results so that the original results page doesn't look drastically different when a user navigates back. It would be off-putting to users, Cramer says, if they had seen a link in the original results that they wanted to click on but, when they went back to the results, found it missing. Therefore, recommended results only appear automatically below the link that was clicked on. "We don't want to jar the users," Cramer says. "[Surf Canyon is] specifically engineered to be as unobtrusive as possible."

Behind the scenes, an algorithm makes the personalization possible. Among other things, the algorithm analyzes which results are clicked, which are ignored, and how much time a user spends looking at the page. Importantly, says Cramer, the algorithm semantically deconstructs a page to determine what it means and how similar it is to others in the results. The results are cumulative: after a couple of clicks, the algorithm can determine if you're most interested in a Canon camera, an SLR camera, or, specifically, a Canon SLR, Cramer says.

Results revealed: A Google search for “thermoelectric cooler” returns 10 standard results. After clicking on the first result and then clicking back to the results page, Surf Canyon shows three suggested results below the first one. Credit: Kate Greene

Marti Hearst, a professor at the School of Information at the University of California, Berkeley, says that Surf Canyon succeeds in presenting the reordered links in a clear, useful, and unobtrusive way. It doesn't require people to do any extra work, as does Google's WikiSearch, a feature that lets users personalize their results by voting them up or down.

However, in her test cases, Hearst found that the algorithm's re-ranked results weren't completely useful. "Where personalization works is where queries are ambiguous," she says, but queries have become increasingly longer over the years, and they tend to provide clues that help the engine disambiguate the results on its own. Additionally, in Hearst's tests of Surf Canyon, she found that it only untangled the different meanings of the acronym ACL (which could mean both anterior cruciate ligament and Association for Computational Linguistics) to a certain point: it kept including mixed results even when she felt that her clicking choices had made it clear that she was interested in the linguistics group.

Cramer and his team say that they have gotten more positive results. In a study they performed, some participants saw a second page of search results that were reordered according to Surf Canyon's algorithm, while others saw a second page with standard results. The researchers found that the participants who had access to reordered results clicked on them 30 to 40 percent more frequently.

What will happen to the ERP market in 2009

Top Five Supply Chain Risk Factors

Top Five Supply Chain Risk Factors

Learn to identify and assess all the risks in your supply chain.

Sunday, February 01, 2009
By  David Blanchard

1. Country of origin: Knowing the location of your supplier's production facilities is important in recognizing their susceptibility to security threats. Countries that are more vulnerable to threats could affect the security of the supplier's plants and could cause delays in your supply chain or compromise the products you receive. A key question to have answered is the physical location of each of the supplier's plants and/or factories, and if possible, the locations of their raw material suppliers.

2. Shipment and delivery accuracy: Ensuring that a supplier can deliver supplies consistently and on time is key to assessing the risk they pose to your supply chain. Be sure to ask for shipment times (daily, weekly, etc.), mode of transportation (air, land or sea), and rerouting procedures when natural disasters interrupt trade lanes.

3. Physical security: Assessing the physical security is very important, especially in countries where terrorism is on the rise. Inquiring about areas such as the materials used to construct the building, existence of a guard gate, adequate lighting around the perimeter, use of locks on all windows and doors, perimeter fencing, and cargo storage procedures will help you assess the ability of the supplier to keep their location secure regardless of natural or institutional threats.

4. Internal processes: Soliciting information about a supplier's internal processes provides visibility into not only the security, but also the controls put in place during the manufacturing process. Suppliers should be asked to explain processes dealing with how keys are checked out, visitors are monitored while on-site, access to cargo is restricted, use of computers and electronic data is controlled, and employee background checks are conducted.

5. Social and environmental responsibilities: Requesting information on the removal of chemicals used during the manufacturing process, or if the supplier abides by the no child labor law are only two of the important questions in this area. It is also instructive to inquire about internal policies such as maternity leave and paid time off as well as the air quality and work environment. Supplier social and environmental responsibilities are becoming a larger factor in assessing risk in a supply chain as product safety regulations continue to be proposed and passed into law.

Source: Integration Point

See Also

• Top Nine Supply Chain Challenges for 2009
• Portrait of Best-In-Class Risk Management

Top Five Supply Chain Risk Factors

Top Five Supply Chain Risk Factors

Learn to identify and assess all the risks in your supply chain.

Sunday, February 01, 2009
By  David Blanchard

1. Country of origin: Knowing the location of your supplier's production facilities is important in recognizing their susceptibility to security threats. Countries that are more vulnerable to threats could affect the security of the supplier's plants and could cause delays in your supply chain or compromise the products you receive. A key question to have answered is the physical location of each of the supplier's plants and/or factories, and if possible, the locations of their raw material suppliers.

2. Shipment and delivery accuracy: Ensuring that a supplier can deliver supplies consistently and on time is key to assessing the risk they pose to your supply chain. Be sure to ask for shipment times (daily, weekly, etc.), mode of transportation (air, land or sea), and rerouting procedures when natural disasters interrupt trade lanes.

3. Physical security: Assessing the physical security is very important, especially in countries where terrorism is on the rise. Inquiring about areas such as the materials used to construct the building, existence of a guard gate, adequate lighting around the perimeter, use of locks on all windows and doors, perimeter fencing, and cargo storage procedures will help you assess the ability of the supplier to keep their location secure regardless of natural or institutional threats.

4. Internal processes: Soliciting information about a supplier's internal processes provides visibility into not only the security, but also the controls put in place during the manufacturing process. Suppliers should be asked to explain processes dealing with how keys are checked out, visitors are monitored while on-site, access to cargo is restricted, use of computers and electronic data is controlled, and employee background checks are conducted.

5. Social and environmental responsibilities: Requesting information on the removal of chemicals used during the manufacturing process, or if the supplier abides by the no child labor law are only two of the important questions in this area. It is also instructive to inquire about internal policies such as maternity leave and paid time off as well as the air quality and work environment. Supplier social and environmental responsibilities are becoming a larger factor in assessing risk in a supply chain as product safety regulations continue to be proposed and passed into law.

Source: Integration Point

See Also

• Top Nine Supply Chain Challenges for 2009
• Portrait of Best-In-Class Risk Management