Introduction to CGI

CGI stands for "Common Gateway Interface". CGI is one method by which a web server can obtain data from (or send data to) databases, documents, and other programs, and present that data to viewers via the web. More simply, a CGI is a program intended to be run on the web. A CGI program can be written in any programming language, but Perl is one of the most popular, and other languages are:

If you're going to create web pages, then at some point you'll want to add a counter, a form to let visitors send you mail or place an order, or something similar. CGI enables you to do that and much more. From mail-forms and counter programs, to the most complex database programs that generate entire websites on-the-fly, CGI programs deliver a broad spectrum of content on the web today.

When a web server gets a request for a static web page, the web server finds the corresponding HTML file on its filesystem. When a web server gets a request for a CGI script, the web server executes the CGI script as another process (i.e., a separate application); the server passes this process some parameters and collects its output, which it then returns to the client just as if had been fetched from a static file.

CGI programming involves designing and writing programs that receive their starting commands from a Web page-usually, a Web page that uses an HTML form to initiate the CGI program. The HTML form has become the method of choice for sending data across the Net because of the ease of setting up a user interface using the HTML Form and Input tags. With the HTML form, you can set up input windows, pull-down menus, checkboxes, radio buttons, and more with very little effort. In addition, the data from all these data-entry methods is formatted automatically and sent for you when you use the HTML form.

CGI programs don't have to be started by a Web page, however. They can be started as the result of a Server Side Include (SSI) execution command. You even can start a CGI program from the command line. But a CGI program started from the command line probably will not act the way you expect or designed it to act. Why is that? Well, a CGI program runs under a unique environment. The WWW server that started your CGI program creates some special information for your CGI program, and it expects some special responses back from your CGI program.

Before your CGI program is initiated, the WWW server already has created a special processing environment for your CGI program in which to operate. That environment includes translating all the incoming HTTP request headers into environment variables that your CGI program can use for all kinds of valuable information. In addition to system information (such as the current date), the environment includes information about who is calling your CGI program, from where your program is being called, and possibly even state information to help you keep track of a single Web visitor's actions. State information is anything that keeps track of what your program did the last time it was called.

Next, the server tries to determine what type of file or program it is calling because it must act differently based on the type of file it is accessing. So, your WWW server first looks at the file extension to determine whether it needs to parse the file looking for SSI commands, execute the Perl interpreter to compile and interpret a Perl program, or just generate the correct HTTP response headers and return an HTML file.

After your server starts up your SSI or CGI program (or even HTML file), it expects a specific type of response from the SSI or CGI program. If your server is just returning an HTML file, it expects that file to be a text file with HTML tags and text in it. If the server is returning an HTML file, the server is responsible for generating the required HTTP response headers, which tell the calling browser the status of the browser's request for a Web page and what type of data the browser will be receiving, among other things.

The SSI file works almost like a regular HTML file. The only difference is that, with an SSI file, the server must look at each line in the file for special SSI commands. If it finds an SSI command, it tries to execute it. The output from the executed SSI command is inserted into the returned HTML file, replacing the special HTML syntax for calling an SSI command. The output from the SSI command will appear within the HTML text just as if it were typed at the location of the SSI command. SSI commands can include other files, execute system commands, and perform many useful functions. The server uses the file extension of the requested Web page to determine whether it needs to parse a file for SSI commands. SSI files typically have the extension .shtml.

If the server identifies the file as an executable CGI program, it executes the program as appropriate. After the server executes your CGI program, your program normally responds with the minimum required HTTP response headers and then some HTML tags. If your CGI program is returning HTML, it should output a response header of Content-Type: text/html. This gives the server enough information to generate any other required HTTP response headers.

---

What is CGI Programming

CGI programming is writing the programs that receive and translate data sent via the Internet to your WWW server. CGI programming is using that translated data and understanding how to send valid HTTP response headers and HTML tags back to your WWW client.

---

Why is it called gateway?

your program acts as a gateway or interface program between other, larger applications. CGI programs often are written in scripting languages such as Perl. Scripting languages really are not meant for large applications. You might create a program that translates and formats the data being sent to it from applications such as online catalogs, for example. This translated data then is passed to some type of database program. The database program does the necessary operations on its database and returns the results to your CGI program. Your CGI program then can reformat the returned data as needed for the Internet and return it to the online catalog customer, thus acting as a gateway between the HTML catalog, the HTTP request/response headers, and the database program.

Alternative Technologies !

There are various alternatives to CGI in which most of them avoid the main drawback to CGI scripts: creating a separate process to execute the script every time it is requested and some of also try to make less of a distinction between HTML pages and code by moving code into HTML pages. Some of major alternatives to CGI are:

• ASP
• PHP
• FastCGI
• mod_perl
• ColdFusion
• Java servlets

---

ASP

ASP stands for Active Server Pages. ASP is a Microsoft Technology that runs inside IIS. IIS stands for Internet Information Services. An ASP file is just the same as an HTML file, it can contain text, HTML, XML, and scripts. Scripts in an ASP file are executed on the server. An ASP file has the file extension &.asp&.

ASP can dynamically edit, change, or add any content of a Web page. Respond to user queries or data submitted from HTML forms. Access any data or databases and return the results to a browser. Customize a Web page to make it more useful for individual users. The advantages of using ASP instead of CGI and Perl, are those of simplicity and speed. Povide security - since ASP code cannot be viewed from the browser.

---

PHP

PHP stands for PHP: Hypertext Preprocessor. It is a server-side scripting language, like ASP. PHP scripts are executed on the server. PHP supports many databases (MySQL, Informix, Oracle, Sybase, Solid, PostgreSQL, Generic ODBC, etc.). PHP is an open source software and is free to download and use. PHP files can contain text, HTML tags and scripts. PHP files are returned to the browser as plain HTML. PHP files have a file extension of ".php", ".php3", or ".phtml".

---

FastCGI

FastCGI is simple because it is actually CGI with only a few extensions:

Like CGI, FastCGI is also language-independent. For instance, FastCGI provides a way to improve the performance of the thousands of Perl applications that have been written for the Web.

Like CGI, FastCGI runs applications in processes isolated from the core Web server, which provides greater security than APIs. (APIs link application code into the core Web server, which means that a bug in one API-based application can corrupt another application or the core server; a malicious API-based application can, for example, steal key security secrets from another application or the core server.)

Although FastCGI cannot duplicate the universality of CGI overnight, the FastCGI developers are committed to propagating FastCGI as an open standard. To that end, free FastCGI application libraries (C/C++, Java, Perl, Tcl) and upgrade modules for popular free servers (Apache, ISS, Lighttpd) are available.

Like CGI, FastCGI is not tied to the internal architecture of any Web server and is therefore stable even when server technology changes. An API reflects the internal architecture of a Web server, so when that architecture changes, so does the API.

---

mod_perl

mod_perl is more than CGI scripting on steroids. It is a whole new way to create dynamic content by utilizing the full power of the Apache web server to create stateful sessions, customized user authentication systems, smart proxies and much more. And your old CGI scripts will continue to work and work very fast indeed.

mod_perl is an optional module for the Apache HTTP server. It embeds a Perl interpreter into the Apache server, so that dynamic content produced by Perl scripts can be served in response to incoming requests, without the significant overhead of re-launching the Perl interpreter for each request.

---

ColdFusion

ColdFusion is the hot way to create dynamic webpages that link to just about any database. ColdFusion is a programming language based on standard HTML (Hyper Text Markup Language) that is used to write dynamic webpages. It lets you create pages on the fly that differ depending on user input, database lookups, time of day or whatever other criteria you dream up! ColdFusion pages consist of standard HTML tags such as , together with CFML (ColdFusion Markup Language) tags such as <CFQUERY>, <CFIF> and <CFLOOP>. ColdFusion was introduced by Allaire in 1996, acquired by Macromedia in a merger in April 2001, and acquired by Adobe in December 2005.

---

Java Servlet

Servlets are the Java platform technology of choice for extending and enhancing Web servers. Servlets provide a component-based, platform-independent method for building Web-based applications, without the performance limitations of CGI programs. And unlike proprietary server extension mechanisms (such as the Netscape Server API or Apache modules), servlets are server- and platform-independent. This leaves you free to select a "best of breed" strategy for your servers, platforms, and tools.

Servlets have access to the entire family of Java APIs, including the JDBC API to access enterprise databases. Servlets can also access a library of HTTP-specific calls and receive all the benefits of the mature Java language, including portability, performance, reusability, and crash protection.

Today servlets are a popular choice for building interactive Web applications. Third-party servlet containers are available for Apache Web Server, Microsoft IIS, and others. Servlet containers are usually a component of Web and application servers, such as BEA WebLogic Application Server, IBM WebSphere, Sun Java System Web Server, Sun Java System Application Server, and others.

Design Sketches

Once you've evolved high-level blueprints and architectural page mockups, you're ready to collaborate with your graphic designer to create deign sketches on paper of major pages in the web site. In the research phase the design team has begun to develop a sense of the desired graphic identity or look and feel. The technical team has assessed the information technology infrastructure of the organization and the platform limitations of the intended audiences. They understand what's possible with respect to features such as dynamic content management and interactivity. And of course the architect has designed the high-level information structure for the site. Design sketches are a great way to pool the collective knowledge of these three teams in a first attempt at interface design for the top level pages of the site. This in a wonderful opportunity for interdisciplinary user interface design using the architectural mocks ups as a guide; the designer begins sketching pates of the site on sheets of paper. As the designer sketches each page questions arise that must be discussed. Here is a sample sketching session dialog:

Programmer: I like what you're doing with the layout of the main page, but I'd like to do something more interesting with the navigation system.

Designer: Can we implement the navigation system using pull down menus? Does that make sense architecturally?

Architect: That might work but it would be difficult to show context in the hierarchy. How about a tear-|way table of contents feature? We've had pretty good reactions to that type of approach front users in the past.

Programmer: We can certainly go with that approach from a purely technical perspective. How would a tear away table of contents look? Can you sketch it for us? I'd like to do a quick and dirty prototype. These sketches allow rapid iteration and intense collaboration.

Architectural Page Mockups

Information architecture blueprints are most useful for presenting a bird’s eye view of the web site. However they to not work well for helping people to envision the contents of any particular page. They are also not straightforward enough for most graphic designers to work from. In Fact no single format perfect job of conveying all aspects of information architecture to all audiences. Because information architectures are multi dimensional, it's important to show them information multiple ways. For these reasons Architectural page mockup are useful tools during conceptual design for complimenting the blueprint view of the site mockups are quick and dirty textual documents that show the content and links of major pages on the website. They enable you to clearly (yet inexpensively) communicate the implications of the architecture at the page level. They are also extremely useful when used in conjunction with scenarios. They help people to see the site in action before any code is written. Finally, they can be employed in some basic usability tests to see if users actually follow the scenarios as you expect. Keep in mind that you only need to mockup major pages of the web site. These mockups and the designs that derive from them can serve as templates for design of subsidiary pages. The mockups are easier to read than blueprints. By integrating aspects of the organizational labeling, and navigation systems in to one view they will help your colleagues to understand the architecture. In laying out the content on a page mockup, you should try to show the logical visual grouping of content items. Placing a content group at the top of the page or using a larger font size indicates the relative importance of that content.

While the graphic designer will make the final and more detailed layout decisions you can make a good start with these mockups.

Conceptual design

Blueprints

What do you mean by blueprint? Blueprints are the architect’s tool of choice for performing the transformation for chaos in to order. Blueprints show the relationship between pages and other content components and can be used to portray organization, navigation and labeling systems. They are often referred to as sitemaps and do in fact have much information common with those supplemental navigation systems. Both the diagram and the navigation system display the shape of the information space information overview, functioning as a condensed map for site developers and users, respectively

High -level Architecture blueprints

High level architecture blueprints are often created by information architects as pat of a top down information architecture process. The very act shaping ideas in to the more structure of a blueprint forces you to become realistic and practical. During the design phase, high level blueprints are most useful for exploring primary organization schemes and approaches. High level blueprints map out the organization and labeling of major areas. Usually beginning with a bird's eye view from the main page of the website.

Creating High -Level Architecture Blue prints

These blueprints can be created by hand, but diagramming software such as Visio or OmniGraffle are preferred. These tools not only help to quickly layout the architecture Blue prints, but can also help with site implementation and administration.

Some Important points:

1).Blueprints focus on major areas and structure of site ignoring many navigation details and page level details.
2).Blueprints are excellent tools for explaining your architectural approaches.
3).Presenting blueprints information person allows you to immediately answer the questions and address client concerns as well as to explore new ideas while they are fresh in your mind and the client's.
4).As you create blueprint it is important to avoid getting locked into a particular type of layout.
5).If a meeting isn't possible, you can accompany blueprints with descriptive test based documents that anticipate and answer the most likely documents.

Keeping Blueprints Simple

As a project moves from strategy to design to implementation, blueprints become more utilitarian. They need be produced and modified quickly and often draw input front increasing number of perspectives, ranging from visual designers to editors to programmers. Those team members need to be able to understand the architecture. So it’s important to develop a simple condensed vocabulary of objects that can explain in a brief legend.

Designing the Search Interface

Concept of Searching system

There are two models of searching systems:

1).In the first and older model user express their information need as query that they enter in a search interface. They may do so using a specialized search language.
2).In the second model users express the information need information the natural language like English.

After this step Queries are matched against an index that represent the site's content and a set of matching documents is identified.

---

Designing the Search Interface

With so much variation among users to account for, there can be no single ideal search interface. Following factors affect choice of search interface:

The levels of searching expertise users have: Are they comfortable with Boolean operators. Or do they prefer natural language? Do they need simple or high powered interface? What about a help page?

The kind of information the user wants: Do they want just a taste or are they doing comprehensive research? Should the results be brief, or should they provide extensive detail for each document?

The type of information being searched is it made up of structured fields or full texts? Is it navigation pages, destination pages, or both? HTML or other formats?

How much information is being searched: will users be overwhelmed by the number of documents retrieved?

---

Support Different Modes of Searching

Use the same interface to allow users to search the product catalog, or the staff directory, or other content areas. Are non-English speakers important to your site? Then provide them with search interfaces in their native languages. Including language specific directions, search commands and operators, and help information. Does your site need to satisfy users with different levels of sophistication with online searching? Then consider making available both a basic search interface and an advanced one.

Simple / Basic search interface

A simple search interface was required; because at limes users wouldn't need all the firepower of an advanced search interface. Especially when conducting simple known item searches. A simple search box is ideal for the novice or for a user with a pretty good sense of what he or she is looking for. Mammal filtering options are provided including searching for keywords within little and abstract fields, searching within the author field or searching within the publication number field. These filtering options provide the user with more power by allowing more specific searching. But because the labels keyword, Author, And publication Number are fairly self explanatory. They don't force the user to think too much about these options.

Advanced search Interface

We needed interface that would accommodate this important expert audience who were used to complex Boolean and proximity operators and who where already very used to the arcane search languages of other commercial information services. This interface supports the following types of searching:

Fielded Searching

Author, keyword, Title, Subject and ten other fields are reachable. A researcher could, for example find a dissertation related to his or her area of interest by searching the subject field, and learn who that doctoral student's advisor was by reading the abstract. To find other related dissertations, the researcher could then search the advisor field to learn about other doctoral students who shared the same advisor.

Familiar Query Language

Because many different query language conventions are supported by traditional on line products, users may be used to an established convention. The effort to support these users is made by allowing variant terms. For the field Degree Date the user can enter either ‘‘ddt’’, ''da'', ''date'', ''Yr '' or year.

Longer Queries

More complex queries often require more space than the single line entry box found in the simple search interface. The more complex interface supports a much longer query.

Reusable Result Sets

Many traditional online information products allow searchers to build sets of results that can be reused. In this example, we've ANDed together the two sets that we've already found and could in turn combine this result with other sets during the iterative process of searching. Because this advanced interface supports so many different types of searching we provided a substantial help page to assist users. For users of common browsers, the help page is launched if a separate browser window so that users don't need to exit the search interface to get help.

---

Searching and browsing systems should be closely integrated

As we mentioned earlier, users typically need to switch back and forth between searching and browsing. In fact users often don't know if they need to search or browse in the first place. Therefore, these respective systems shouldn't live in isolation from one another. The search/browse approach can be extended by making search and browse options available on the search result page as well, especially on null results pages when a user might be at a dead end and needs to be gently led back to the process of iterative searching and browsing before frustration sets in.

Searching should conform to the site's Look and feel

Search engine interfaces and more importantly, retrieval results, should look and behave like the rest of your site.

Search Options Should Be Clear

We all pay lip service to the need for user documentation, but with searching it's really a must Because, so many different variables are involved with searching there are many opportunities for things to go wrong on a help or Documentation page consider letting the user know the fallowing:

What is being searched?

Users often assume that their search query is being run against the full test of every page in your site Instead your site may support fielded searching or another type of selective searching. If they're curious users should be able to find out exactly what they are searching.

How they can formulate search queries

What good is it to build in advanced querying capabilities if the user never knows about them? Shows off the power of your search engine with excellent real life examples. In other words make sure your examples actually work and retrieve relevant documents if the user decides to test them.

User options

Can the user do other neat things much as changing the sorting order of retrieval results? Show them off as well!

What to do if the user can’t find the right information
It is important to provide the user with some tricks to handle the following three situations:

• I'm getting too much stuff
• I'm not getting anything
• I'm getting the wrong stuff

For case (a), you might suggest approaches that narrow the retrieval results. For example if your system suppers the Boolean operator AND, suggest that users combine multiple search terms with an AND between them (ANDing together terms reduces retrieval size).

If they are retrieving zero 'results as in case (b), suggest the operator OR the use of multiple search terms the use of truncation (which wife retrieve a term's use o variants), and so on.

If they are completely dissatisfied with their searches, case(c), you might suggest that they contact someone who knows the site’s content directly for custom assistance, it may be a resource intensive approach, but it’s a far superior last resort to ditching the user without helping them at all.

---

Choose a search Engine That Fits Users' Needs

At this point, you ideally will know something about the sorts of searching capabilities that your site’s users will require. So select a search engine that satisfies those needs as much as possible for example, if you know that your site’s users already very familiar with a particular way of specifying a query such as the use of operators, then the search engine you choose should also support using Boolean operators. Does the size of your site suggest that users will get huge retrieval results? Be sure that your engine be supports techniques for whittling down retrieval sizes, such as the AND & NOT operators , or that it supports relevance ranked results that list the most relevant results at the top will users have a problem with findings the right terms to use in their search queries?

---

Display search Results sensibly

You can configure how your search engine displays search results information many ways. How you configure your search engine results depends on two factors.

The first factor is the degree of structure your content has. What will your search engine be able to display besides just the titles of retrieved documents? Is your site's content sufficiently structured so that the engine can parse out and display such information as an author, a date an abstract, and so on?

The other factor is what your site's users really want. What sorts of information do they need and expect to be provided as they review search results?

When you are configuring the way your search engine displays results you should consider these issues:
1) How much information should be displayed for each retrieved document?

To display less information per result when you anticipate large result sets. This will shorten the length of the results page making it easier to read. To display less information to users who know what they're looking for, and more information to users who aren't sure what they want.

2).What Information should be displayed for each retrieved document?

Which fields you show for each document obviously depends on which fields are available in each document, what your engine displays also depends on how the content is to be used. Users of phone directories for example want phone numbers first and foremost. So it makes sense to show them the information from the phone number field on the results page. Lastly, the amount of space available on a page is limited: You can't have each field displayed, so you should choose carefully and use the space that is available wisely.

3).How many retrieved documents should be displayed?

How many documents are displayed depends on the preceding two factors: If your engine displays a lot of Information for each retrieved document, you'll want to consider a smaller size for the retrieval set, and vice versa. Additionally the user's monitor resolution and browser settings will affects the amount of information that can be displayed individually.

4).How should retrieved document be sorted?
Common options or sorting retrieval results include:

• In chronological order.
• Alphabetically by title, author, or other fiends.
• By an odd thing called relevance.

Certainly, if your site is providing access to press releases or other news- oriented information, sorting by reverse chronological order makes good sense. Chronological order is less common, and can be useful for presenting historical data.

Alphabetical sorts are a good general purpose sorting approach (most users are familiar with the order of the alphabet). Alphabetical sorting works best if initial articles such as a and the are omitted from the sort order (certain search engines provide this option).

Relevance is an interesting concept; when a search engine retrieves 2000 documents, is not it great to have them sorted with the most relevant at the top, and the least relevant at the bottom? Relevance ranking algorithms are typically determined by some combination of the following; how many of the query's terms occur in the retrieved document; how many times terms occur in that document; how close to each other those terms occur and where the terms occur.

---

Always provide the user with feedback

When a user executes a search, he or she expects result. Usually a query with retrieves at least one document, so the user's expectation is fulfilled. But sometimes a search retrieves zero results. Let the user know by creating a different results page especially for these cases. This page should make it painfully clear that nothing was retrieved, and give an explation as to why, tips for improving retrieval results and links to both the help area and to a new search interface so the user can try again.

---

Other Considerations

You might also consider including a few easy to implement but very useful things in your engine's search results:

Repeat back the original search query prominently on the results Page

As users browse through search results, they may forget what they searched for in the first place remind them. Also include the query in the page titles; this will make it easier for users to find it in their browser's history lists.

Let the user know how many document in total were retrieved.

Users want to know how many documents have been retrieved before they begin reviewing the results. Let them know: if the number is too large, they should have the option to refine their search.

Let the user know where he or she is in the current retrieved set.

It's helpful to let users know that they're viewing documents 31-40 of the 83 total that they've retrieved.

Always make it easy for the user to revise a search or sort a new One.

Give them these options on every results page and display the current search query on the revise search page so they can modify it without reentering it.