Guide - Cookies-Part.1 Guide [PR.157]

computing - www.mjo.co.uk

DOCUMENTATION - COOKIES

Introduction
This document was written to support the Cookie Script written to evaluate Cookies and Cookie behaviour.

Cookies
Cookies, have now been around for some considerable time, and my involvement with them comes from a time around 1995 when Netscape Navigator was King, and sadly - to my mind - most of the Browsers of today lack the intuitive styling that these older Browsers possessed, and despite going forward, I think in many repects we have gone backwards regarding them, however, I digress ...
Cookies, in many cases are still not very well understood. Many people have misconceptions, ranging from Cookies being Devilish or Dangerous or they are Viruses or Spyware, totally misunderstanding what they are, and how they work.

Background
Lou Montulli an employee of Netscape Communications, is credited with applying the concept of - what was then known as - the 'Magic Cookie' to web communication in 1994. The Netscape Navigator - my favourite being 3 - supported cookies since its first version, but, Cookies are now supported by all major web browsers, and play a very important role as part of web development, without which today, many important web applications would be rendered useless. Should any further reading be required, a quick internet search will turn up the 'Original Cookie Specification' which has actually not changed very much since its inception!

What is a Cookie?
A Cookie is quite simply a small text file that is stored by the Browser on the users machine - one of the reasons that struck fear into the heart of many a user! - they are formed of plain text, and contain no executable code whatsoever. A Web Page or Server instructs the Browser to store this information, which is looked at with each subsequent request, based on a set of rules. Web servers can then use this information to identify individual user requests, and as this is just text data, it isn't harmful in any respect, either in it's content, or of itself.
BUT! - and it is quite a significant 'BUT' - they can be Classed as being very 'INTRUSIVE'

Cooke creation
A web server specifies a cookie to be stored by sending an HTTP header called Set-Cookie. The format of the Set-Cookie header is a simple string as follows:

Set-Cookie: value; expires=date; domain=domain; path=path; secure

The first part of the header is: 'The Value' which is typically a string in the format of a Name=Value Pair, such as that used in the Query String!
The options specified with Set-Cookie are for the browser's use only, and cannot be retrieved once they have been set. If there are multiple cookies for the given request, then they are separated by a semicolon and space, such as:

Cookie: value1; value2 etc ...

Server-Side frameworks normally provide functionality to parse cookies to make their values available.

The Options
Each of the options after the cookie value are separated by a semicolon and space and each specifies rules about when the cookie should be sent back to the server, we will now consider each option in turn.

The - Expires Option
Indicates when the cookie should no longer be consider valid and sent to the server and therefore may be deleted by the browser. The value for this option is a date in the format:

Wdy, DD-Mon-YYYY HH:MM:SS GMT

Note: This is the standard JavaScript GMT/UTC Format which can be used directly, as in the 'JavaScript Cookie Script'
such as:

Set-Cookie: name=MjO; expires=Sat, 18 May 2018 14:00:00 GMT

Without the expires option, a cookie has a lifespan of a single session. A session is defined as finished when the browser is shut down, so session cookies exist only while the browser remains open, else the lifespan of the Cookie is given by this value!
If the expires option is set to a date in the past, then the cookie is no longer considered valid, and is immediately deleted.

The - Domain Option
The next option is domain, which indicates the domain(s) for which the cookie should be sent. By default, domain is set to the host name of the page setting the cookie, so the cookie value is sent whenever a request is made to the same host name. Eg. If the default domain for a cookie set on this site would be www.mjo.co.uk.

Set-Cookie: name=MjO; domain=mjo.co.uk;

NB. The value set for the domain option must be part of the host name that is sending the Set-Cookie header.

The - Path Option
Another way to control when the Cookie header will be sent is to specify the path option. Similar to the domain option, path indicates a URL path that must exist in the requested resource before sending the Cookie header. This comparison is done by comparing the option value character-by-character against the start of the request URL. If the characters match, then the Cookie header is sent. For instance:

Set-Cookie: name=MjO; path=/tutor

In this example, the path option would match /tutor, /tutorial, etc.; anything that begins with /tutor is valid. Note that this comparison is only done once the domain option has been verified. The default value for the path option is the path of the URL that sent the Set-Cookie header.

The - Secure Option
The last option is secure. Unlike the other options, this is just a flag and has no additional value specified. A secure cookie will only be sent to the server when a request is made using SSL and the HTTPS protocol. The idea that the contents of the cookie are of high value and could be potentially damaging to transmit as clear text. Sample:

Set-Cookie: name=MjO; secure

In reality, confidential or sensitive information should never be stored or transmitted in cookies as the entire mechanism is inherently insecure. By default, cookies set over an HTTPS connection are automatically set to be secure.

Cookie maintenance and lifecycle
Any number of options can be specified for a single cookie, and those options may appear in any order. For example:

Set-Cookie: name=MjO; domain=mjo.co.uk; path=/tutor

This cookie has three identifying characteristics: the cookie name, the domain, the path, and the secure flag. In order to change the value of this cookie in the future, another Set-Cookie header must be sent using the same cookie name, domain, and path. Changing even one of these options creates a completely different cookie.
When a cookie is created with an expiration date, that expiration date relates to the cookie identified by it's full data. In order to change the expiration date of a cookie, you must specify the exact same data. When changing a cookies value, you need not set the expiration date each time because it's not part of the identifying information. For Example:

Set-Cookie: name=MjO; expires=Sat, 18 May 2018 14:00:00 GMT

The expiration date of the cookie has now been set, so the next time I want to change the value of the cookie, I can just use its name, in fact, the expiration date won't change until it's manually changed.
NB. Please be mindful that the expiration date is checked against the system time on the computer running the browser, there is no way to verify that the system time is in sync with server time, so errors may well occur when there is a discrepancy between the two!

Set-Cookie: name=MjO

Automatic Cookie Removal
There are several reasons why a cookie might be automatically removed by the browser:

Session Cookies are removed when the session is over (When the Browser is Closed).
Persistent Cookies are removed when the Expiration Date/Time is Reached!
If the Browsers Cookie limit is reached, Cookies will be removed to make room for the most recent.

NB. Cookie management is important to avoid removal in cases where it it unintended, such as the above

Cookie restrictions
There are restrictions placed on cookies in order to prevent abuse and protect both the browser and the server ie. The number of Cookies/Domain and Cookie size, and care must be observed in the case of the latter!

Subcookies
Due to the cookie number limit, developers have come up with the idea of subcookies to increase the amount of storage available to them. Subcookies are name-value pairs stored within a cookie value and typically have a format similar to below:

name=value&scn1=scv1&scn2=scv2&scn3=scv3

This approach allows a single cookie to hold multiple name/value pairs without going over the browsers Cookie limit. The downside to creating cookies in this format is that custom parsing is needed to extract the values rather than relying on the much simplier cookie format. Some server-side frameworks are beginning to support subcookie storage.

Cookies in JavaScript
As can be seen from above, one of the reasons I wrote this article was to support the JavaScript Cookie Evaluation Script which can seen Here which can be used to fully investigate Cookie behavour in terms of Creation, Manipulation and the Removal of Cookies in JavaScript by using the document.cookie property. This property acts as the Set-Cookie header when assigned to, and as the Cookie header when read from. When creating a cookie, you must use a string that follows the same format that Set-Cookie expects:

document.cookie="name=MjO; domain=mjo.co.uk; path=/tutor";

Setting the value of document.cookie does not delete all of the cookies stored on the page. It simply creates or modifies the cookie specified in the string. The next time a request is made to the server, these cookies are sent along with any others that were created via Set-Cookie. There is no perceivable difference between these cookies.
To retrieve cookie values in JavaScript, you just need to read from the document.cookie property. The returned string is in the same format as the Cookie header value, so multiple cookies are separated by a semicolon and space.
Example:

name1=MjO; name2=WebMaster

Because of this, you need to parse the cookie string manually to extract actual cookie data. Inspection of the Script mentioned above will allow the user to fully investigate Cookie functionality!
Cookies returned by accessing document.cookie follow the same access rules as cookies sent to the server. In order to access cookies via JavaScript, the page must be in the same domain and have the same path and have the same security level as specified by the cookie.
NB. The options as mentioned above are for the Browsers use only, so you cannot retrieve the domain, path, expires date, or the state of the secure flag once a Cookie has been set via JavaScript!

Conclusion
There is a great deal to know and understand regarding the creation and use of Cookies in order to use them effectively, and I do hope that this document, and the Evaluation Script, has provided at least some basic enlightenment regarding them!

computing - www.mjo.co.uk

DOCUMENTATION - COOKIES Introduction This document was written to support the Cookie Script written to evaluate Cookies and Cookie behaviour. Cookies Cookies, have now been around for some considerable time, and my involvement with them comes from a time around 1995 when Netscape Navigator was King, and sadly - to my mind - most of the Browsers of today lack the intuitive styling that these older Browsers possessed, and despite going forward, I think in many repects we have gone backwards regarding them, however, I digress ... Cookies, in many cases are still not very well understood. Many people have misconceptions, ranging from Cookies being Devilish or Dangerous or they are Viruses or Spyware, totally misunderstanding what they are, and how they work. Background Lou Montulli an employee of Netscape Communications, is credited with applying the concept of - what was then known as - the 'Magic Cookie' to web communication in 1994. The Netscape Navigator - my favourite being 3 - supported cookies since its first version, but, Cookies are now supported by all major web browsers, and play a very important role as part of web development, without which today, many important web applications would be rendered useless. Should any further reading be required, a quick internet search will turn up the 'Original Cookie Specification' which has actually not changed very much since its inception! What is a Cookie? A Cookie is quite simply a small text file that is stored by the Browser on the users machine - one of the reasons that struck fear into the heart of many a user! - they are formed of plain text, and contain no executable code whatsoever. A Web Page or Server instructs the Browser to store this information, which is looked at with each subsequent request, based on a set of rules. Web servers can then use this information to identify individual user requests, and as this is just text data, it isn't harmful in any respect, either in it's content, or of itself. BUT! - and it is quite a significant 'BUT' - they can be Classed as being very 'INTRUSIVE' Cooke creation A web server specifies a cookie to be stored by sending an HTTP header called `Set-Cookie`. The format of the `Set-Cookie` header is a simple string as follows: `Set-Cookie: value; expires=date; domain=domain; path=path; secure` The first part of the header is: 'The Value' which is typically a string in the format of a Name=Value Pair, such as that used in the Query String! The options specified with `Set-Cookie` are for the browser's use only, and cannot be retrieved once they have been set. If there are multiple cookies for the given request, then they are separated by a semicolon and space, such as: `Cookie: value1; value2 etc ...` Server-Side frameworks normally provide functionality to parse cookies to make their values available. The Options Each of the options after the cookie value are separated by a semicolon and space and each specifies rules about when the cookie should be sent back to the server, we will now consider each option in turn. The - Expires Option Indicates when the cookie should no longer be consider valid and sent to the server and therefore may be deleted by the browser. The value for this option is a date in the format: `Wdy, DD-Mon-YYYY HH:MM:SS GMT` Note: This is the standard JavaScript GMT/UTC Format which can be used directly, as in the 'JavaScript Cookie Script' such as: `Set-Cookie: name=MjO; expires=Sat, 18 May 2018 14:00:00 GMT` Without the `expires` option, a cookie has a lifespan of a single session. A session is defined as finished when the browser is shut down, so session cookies exist only while the browser remains open, else the lifespan of the Cookie is given by this value! If the `expires` option is set to a date in the past, then the cookie is no longer considered valid, and is immediately deleted. The - Domain Option The next option is `domain`, which indicates the domain(s) for which the cookie should be sent. By default, `domain` is set to the host name of the page setting the cookie, so the cookie value is sent whenever a request is made to the same host name. Eg. If the default domain for a cookie set on this site would be `www.mjo.co.uk`. `Set-Cookie: name=MjO; domain=mjo.co.uk;` NB. The value set for the `domain` option must be part of the host name that is sending the `Set-Cookie` header. The - Path Option Another way to control when the `Cookie` header will be sent is to specify the `path` option. Similar to the domain option, `path` indicates a URL path that must exist in the requested resource before sending the `Cookie` header. This comparison is done by comparing the option value character-by-character against the start of the request URL. If the characters match, then the `Cookie` header is sent. For instance: `Set-Cookie: name=MjO; path=/tutor` In this example, the `path` option would match `/tutor`, `/tutorial`, etc.; anything that begins with `/tutor` is valid. Note that this comparison is only done once the `domain` option has been verified. The default value for the `path` option is the path of the URL that sent the `Set-Cookie` header. The - Secure Option The last option is `secure`. Unlike the other options, this is just a flag and has no additional value specified. A secure cookie will only be sent to the server when a request is made using SSL and the HTTPS protocol. The idea that the contents of the cookie are of high value and could be potentially damaging to transmit as clear text. Sample: `Set-Cookie: name=MjO; secure` In reality, confidential or sensitive information should never be stored or transmitted in cookies as the entire mechanism is inherently insecure. By default, cookies set over an HTTPS connection are automatically set to be secure. Cookie maintenance and lifecycle Any number of options can be specified for a single cookie, and those options may appear in any order. For example: `Set-Cookie: name=MjO; domain=mjo.co.uk; path=/tutor` This cookie has three identifying characteristics: the cookie `name`, the `domain`, the `path`, and the `secure` flag. In order to change the value of this cookie in the future, another `Set-Cookie` header must be sent using the same cookie name, domain, and path. Changing even one of these options creates a completely different cookie. When a cookie is created with an expiration date, that expiration date relates to the cookie identified by it's full data. In order to change the expiration date of a cookie, you must specify the exact same data. When changing a cookies value, you need not set the expiration date each time because it's not part of the identifying information. For Example: `Set-Cookie: name=MjO; expires=Sat, 18 May 2018 14:00:00 GMT` The expiration date of the cookie has now been set, so the next time I want to change the value of the cookie, I can just use its name, in fact, the expiration date won't change until it's manually changed. NB. Please be mindful that the expiration date is checked against the system time on the computer running the browser, there is no way to verify that the system time is in sync with server time, so errors may well occur when there is a discrepancy between the two! `Set-Cookie: name=MjO` Automatic Cookie Removal There are several reasons why a cookie might be automatically removed by the browser: Session Cookies are removed when the session is over (When the Browser is Closed). Persistent Cookies are removed when the Expiration Date/Time is Reached! If the Browsers Cookie limit is reached, Cookies will be removed to make room for the most recent. NB. Cookie management is important to avoid removal in cases where it it unintended, such as the above Cookie restrictions There are restrictions placed on cookies in order to prevent abuse and protect both the browser and the server ie. The number of Cookies/Domain and Cookie size, and care must be observed in the case of the latter! Subcookies Due to the cookie number limit, developers have come up with the idea of subcookies to increase the amount of storage available to them. Subcookies are name-value pairs stored within a cookie value and typically have a format similar to below: `name=value&scn1=scv1&scn2=scv2&scn3=scv3` This approach allows a single cookie to hold multiple name/value pairs without going over the browsers Cookie limit. The downside to creating cookies in this format is that custom parsing is needed to extract the values rather than relying on the much simplier cookie format. Some server-side frameworks are beginning to support subcookie storage. Cookies in JavaScript As can be seen from above, one of the reasons I wrote this article was to support the JavaScript Cookie Evaluation Script which can seen Here which can be used to fully investigate Cookie behavour in terms of Creation, Manipulation and the Removal of Cookies in JavaScript by using the `document.cookie` property. This property acts as the `Set-Cookie` header when assigned to, and as the `Cookie` header when read from. When creating a cookie, you must use a string that follows the same format that `Set-Cookie` expects: `document.cookie="name=MjO; domain=mjo.co.uk; path=/tutor";` Setting the value of `document.cookie` does not delete all of the cookies stored on the page. It simply creates or modifies the cookie specified in the string. The next time a request is made to the server, these cookies are sent along with any others that were created via `Set-Cookie`. There is no perceivable difference between these cookies. To retrieve cookie values in JavaScript, you just need to read from the `document.cookie` property. The returned string is in the same format as the `Cookie` header value, so multiple cookies are separated by a semicolon and space. Example: `name1=MjO; name2=WebMaster` Because of this, you need to parse the cookie string manually to extract actual cookie data. Inspection of the Script mentioned above will allow the user to fully investigate Cookie functionality! Cookies returned by accessing `document.cookie` follow the same access rules as cookies sent to the server. In order to access cookies via JavaScript, the page must be in the same domain and have the same path and have the same security level as specified by the cookie. NB. The options as mentioned above are for the Browsers use only, so you cannot retrieve the `domain, path, expires` date, or the state of the `secure` flag once a Cookie has been set via JavaScript! Conclusion There is a great deal to know and understand regarding the creation and use of Cookies in order to use them effectively, and I do hope that this document, and the Evaluation Script, has provided at least some basic enlightenment regarding them!