Trouble In the House of Google

2011-03-21T17:02:00.000+05:00

Let's look at where stackoverflow.com traffic came from for the year of 2010.

When 88.2% of all traffic for your website comes from a single source, criticizing that single source feels … risky. And perhaps a bit churlish, like looking a gift horse in the mouth, or saying something derogatory in public about your Valued Business Partnertm.

Still, looking at the statistics, it's hard to avoid the obvious conclusion. I've been told many times that Google isn't a monopoly, but they apparently play one on the internet. You are perfectly free to switch to whichever non-viable alternative web search engine you want at any time. Just breathe in that sweet freedom, folks.

Sarcasm aside, I greatly admire Google. My goal is not to be acquired, because I'm in this thing for the long haul – but if I had to pick a company to be acquired by, it would probably be Google. I feel their emphasis on the information graph over the social graph aligns more closely with our mission than almost any other potential suitor I can think of. Anyway, we've been perfectly happy with Google as our de-facto traffic sugar daddy since the beginning. But last year, something strange happened: the content syndicators began to regularly outrank us in Google for our own content.

Syndicating our content is not a problem. In fact, it's encouraged. It would be deeply unfair of us to assert ownership over the content so generously contributed to our sites and create an underclass of digital sharecroppers. Anything posted to Stack Overflow, or any Stack Exchange Network site for that matter, is licensed back to the community in perpetuity under Creative Commons cc-by-sa. The community owns their contributions. We want the whole world to teach each other and learn from the questions and answers posted on our sites. Remix, reuse, share – and teach your peers! That's our mission. That's why I get up in the morning.

However, implicit in this strategy was the assumption that we, as the canonical source for the original questions and answers, would always rank first. Consider Wikipedia – when was the last time you clicked through to a page that was nothing more than a legally copied, properly attributed Wikipedia entry encrusted in advertisements? Never, right? But it is in theory a completely valid, albeit dumb, business model. That's why Joel Spolsky and I were confident in sharing content back to the community with almost no reservations – because Google mercilessly penalizes sites that attempt to game the system by unfairly profiting on copied content. Remixing and reusing is fine, but mass-producing cheap copies encrusted with ads … isn't.

I think of this as common sense, but it's also spelled out explicitly in Google's webmaster content guidelines.

However, some webmasters attempt to improve their page's ranking and attract visitors by creating pages with many words but little or no authentic content. Google will take action against domains that try to rank more highly by just showing scraped or other auto-generated pages that don't add any value to users. Examples include:
Scraped content. Some webmasters make use of content taken from other, more reputable sites on the assumption that increasing the volume of web pages with random, irrelevant content is a good long-term strategy. Purely scraped content, even from high-quality sources, may not provide any added value to your users without additional useful services or content provided by your site. It's worthwhile to take the time to create original content that sets your site apart. This will keep your visitors coming back and will provide useful search results.

In 2010, our mailboxes suddenly started overflowing with complaints from users – complaints that they were doing perfectly reasonable Google searches, and ending up on scraper sites that mirrored Stack Overflow content with added advertisements. Even worse, in some cases, the original Stack Overflow question was nowhere to be found in the search results! That's particularly odd because our attribution terms require linking directly back to us, the canonical source for the question, without nofollow. Google, in indexing the scraped page, cannot avoid seeing that the scraped page links back to the canonical source. This culminated in, of all things, a special browser plug-in that redirects to Stack Overflow from the ripoff sites. How totally depressing. Joel and I thought this was impossible. And I felt like I had personally failed all of you.

The idea that there could be something wrong with Google was inconceivable to me. Google is gravity on the web, an omnipresent constant; blaming Google would be like blaming gravity for my own clumsiness. It wasn't even an option. I started with the golden rule: it's always my fault. We did a ton of due diligence on webmasters.stackexchange.com to ensure we weren't doing anything overtly stupid, and uber-mensch Matt Cutts went out of his way to investigate the hand-vetted search examples contributed in response to my tweet asking for search terms where the scrapers dominated. Issues were found on both sides, and changes were made. Success!

Despite the semi-positive resolution, I was disturbed. If these dime-store scrapers were doing so well and generating so much traffic on the back of our content – how was the rest of the web faring? My enduring faith in the gravitational constant of Google had been shaken. Shaken to the very core.

Throughout my investigation I had nagging doubts that we were seeing serious cracks in the algorithmic search foundations of the house that Google built. But I was afraid to write an article about it for fear I'd be claimed an incompetent kook. I wasn't comfortable sharing that opinion widely, because we might be doing something obviously wrong. Which we tend to do frequently and often. Gravity can't be wrong. We're just clumsy … right?

I can't help noticing that we're not the only site to have serious problems with Google search results in the last few months. In fact, the drum beat of deteriorating Google search quality has been practically deafening of late:

Anecdotally, my personal search results have also been noticeably worse lately. As part of Christmas shopping for my wife, I searched for "iPhone 4 case" in Google. I had to give up completely on the first two pages of search results as utterly useless, and searched Amazon instead.

People whose opinions I respect have all been echoing the same sentiment -- Google, the once essential tool, is somehow losing its edge. The spammers, scrapers, and SEO'ed-to-the-hilt content farms are winning.

Like any sane person, I'm rooting for Google in this battle, and I'd love nothing more than for Google to tweak a few algorithmic knobs and make this entire blog entry moot. Still, this is the first time since 2000 that I can recall Google search quality ever declining, and it has inspired some rather heretical thoughts in me -- are we seeing the first signs that algorithmic search has failed as a strategy? Is the next generation of search destined to be less algorithmic and more social?

It's a scary thing to even entertain, but maybe gravity really is broken. Posted by Jeff Atwood View blog reactions

I'm pretty sure I would buy a stress ball shaped like your head.

William Stevenson on January 3, 2011 3:41 AM

Anecdotally I had the same problem the other day. Horror of horrors Bing was better in the end!

Alexandronov on January 3, 2011 3:47 AM

This is a really interesting article and reflects what I have been feeling for a while, that the relentless and exponential rise in SEO activity would eventually start to affect the usefulness of Google.
In a weird way it feels analogous to when people say that Windows gets more virus attacks because it has a bigger audience than other operating systems rather than being less secure necessarily (Im not saying it isn't)
In other words, perhaps Bing has an advantage here as it is less targeted by SEO activity, the same way that non-Windows operating systems dont suffer the same level of virus attacks?

Barry Lloyd on January 3, 2011 3:53 AM

You’re confusing a model with reality: Google isn’t gravity, they’re trying to *model* it using their ranking algorithm. And as your experience clearly shows, it is currently modelling gravity (i.e. the “reality” of which sites are relevant and which aren’t) badly.

To stay in your metaphor, it’s clearly time for a paradigm shift, a kind of Einstein of web ranking algorithms. Sounds like a very interesting thesis topic. cstheory.SE, anyone?

Konrad on January 3, 2011 3:53 AM

I recall that for a while mirror sites started edging out Wikipedia. Google apparently added a bonus for Wikipedia just to force it to the top over its clones. Is it reasonable to expect them to do that for everyone? I don't know.

Peter Da Silva on January 3, 2011 3:58 AM

This is exactly the reason why I supported webmasters.se so adamantly. Where else could someone go to get great expert help for problems like this?

I've often wondered about the case of Wikipedia, and how scrapers that simply syndicate it are quickly penalized. I think Google has _some_ sort of 'special intervention' when it comes to Wikipedia. I just hope they afford the same luxury to Stack Exchange.

Tinkertim on January 3, 2011 4:06 AM

I have been frustrated with this in recent months also. A scraping website has recently copied the top posts from my blog verbatim without even attributing the original. Google somehow ranks the copies with a higher page rank, causing a significant drop in my stats.

The worst thing is that you are mostly powerless in this situation; who do you complain to, google?

Meekrosoft.wordpress.com on January 3, 2011 4:06 AM

I was recently shown an application for blind testing search engines ( http://blindsearch.fejus.com/ ), and was surprised to find that Bing and Yahoo often delivered better results than Google.

Krisjoh on January 3, 2011 4:11 AM

There are search terms that do not list Wikipedia content on the first page, but scraped content as the first result, e.g. http://www.google.com/search?q=elvett+semic

I have tested it from Germany, if it makes any difference.

Residuum on January 3, 2011 4:13 AM

It is because contrary to wikipedia people don't link back to StackExchange, they just want the info and close the tab, so the original content provider doesn't gain anything.

But we should try to find why spammers are better than you, what are they doing better ? You should ask them.

Hokkos on January 3, 2011 4:23 AM

In a techcrunch post (http://techcrunch.com/2011/01/01/why-we-desperately-need-a-new-and-better-google-2), Vivek Wadhwa said the same thing. Trouble over google head ?

Tyseo on January 3, 2011 4:55 AM

This is a really interesting article. More and more recently these sites are taking priority but I have only seen it with stack overflow related content. I always end up ignoring them and trying to find the stack overflow link as ultimately the number of adverts show you the content didn't originate on that site but I think as other people have said it would be interesting to find out why these sites are taking priority.

Scsmith on January 3, 2011 4:56 AM

Wait - "broken" doesn't mean "broken beyond repair", let alone that somebody else would be able to provide a better fix.

At first sight this is merely a problem of Google neglecting to use a specific bit of information in their filtering, namely the information who copied content from who. It won't be possible to do this automatically with 100% accuracy, but I guess if you take all content from two webpages as prepared for indexing (that is, basically the plaintext conversion), compare them for similarity, and when found to be similar, downgrade the ranking of the one to appear most recently (which is not so very easy to determine) then you can mostly fix this problem.

The basic problem I see is that even with smart optimization techniques (e.g. limit comparisons to pages with similar sets of keywords) the similarity testing required probably won't scale. However there are ways around that too (e.g. don't calculate it at indexing time but spread the calculation over queries at querying time, to make ranking progressively smarter, but I have no idea if Google's infrastructure allows stuff like this).

reinierpost on January 3, 2011 4:59 AM

What we need is a quality measure for search result and this can only be provided by the user. The best way to collect user feedback is by making the process social.

Google previous vote up/down is lacking in social interaction, it could for example shows how many users vote up/down for a certain site.

Charles Gunawan on January 3, 2011 5:10 AM

It's something annoying indeed. There will always be ways of cheating the system, but I think Google should close the holes in a faster pacing.

Leniel Macaferi on January 3, 2011 5:12 AM

Google should buy delicious and add relevance based on number of bookmarks and it's tags. =)
When search results show only a bunch of add based with no real content sites I go to delicious and the same search most of the time brings helpful results.

Iraê Lambert de Carvalho on January 3, 2011 5:13 AM

Thank you for this. For near two years I've seen an increasing deterioration in the usefulness of Google results and thought that was what was happening to the web in general ("blogging is dead," for example). The few times I used another search engine, the results were better and that confused me, given my past experience with how good Google once was. At least now I know it's not the web in general and it's not been my imagination: It's been Google.

twitter.com/mikecane on January 3, 2011 5:25 AM

I used to work at a search engine company, so I know this is a very difficult problem, and I honestly do believe Google is trying their hardest, but clearly not enough, and you have to wonder if it's partly because they are making money in many cases from those scraper sites (IE they have Google ads).

My site - http://www.ausedcar.com used to be in the top 10 on Google for the very popular keywords "used cars" but I've been knocked back to around #45 over the years simply because everyone in front of me uses black hat SEO. Including major million dollar companies! The only difference is that Google is not going to blacklist a major company, while they probably would blacklist me, so there is essentially no way for the little guy to do anymore then hope Google will give some scraps off the table.

The fact that you simply can't do a product review search on Google anymore without getting spam is serious trouble for Google. Unfortunately Bing seems to copy the same algorithm, if they were the "Anti-spam" engine, they could gain real ground.

Phil Anderson on January 3, 2011 5:26 AM

There's one thing that I never understand by any critic of Google (especially in the search domain), when you say:

"Still, looking at the statistics, it's hard to avoid the obvious conclusion. I've been told many times that Google isn't a monopoly, but they apparently play one on the internet. You are perfectly free to switch to whichever non-viable alternative web search engine you want at any time. Just breathe in that sweet freedom, folks."

What do you mean?

Do you expect that a better search engine just materialize out of thin air? What are you expecting? That perhaps Google should fund their competition? Maybe they should hold back a bit on improving their technology and let the other poor souls catch up?

One thing I dislike very much is when people feel entitled, like it's their birthright, to something they should work hard for. Have all the free internet services gotten you into the mindset of a spoiled consumer? This is not how entrepreneur thinks, this kind of thinking is not going to make the world go round.

Yes, you are right, you are being "churlish", show me a better search engine and then we'll talk.

Victorbstan on January 3, 2011 5:28 AM

I'm a member of StackOverflow, but I got into the habit of using Google to search for results on StackOverflow. When I remember, I just scope the search to limit to only the stackoverlow.com site.

But, as your Amazon search experience, I'd just search StackOverflow directly if I felt the search results would be as good as Google. (I haven't compared recently)

Bikeoid on January 3, 2011 5:30 AM

Moving away from content-based ranking feels scary to me. I'd rather have things stay as they are. Or, if you will, give the social search engine as an optional approach to enrich the algorithmic search.

I feel the problem with Google however is not the algorithms, but the absence of essential information that can no longer be ignored; i.e. Google has to stop presenting results as a veritable shopping list and seriously consider the introduction of categories into its search engine.

This much was attempted by Cuil and was my favorite feature of that otherwise failed attempt at producing an alternative to Google. Backed up by intelligent algorithms like Google is capable of doing, scrapers wouldn't be able to avoid being moved to their own category away from normal searches.

Mario Figueiredo on January 3, 2011 5:39 AM

It seems to happen every year in the leadup to christmas when companies need to rank well (and will pay for adwords if they aren't ranking)

Ilikeprivacy on January 3, 2011 5:43 AM

I remember very well that i asked some friends if they've noticed how google results deteriorated the day google instant search was deployed.
Another clear difference is in the way apple sdk documentation don't appear anymore among google search results ( although in that case it may very well be apple's decision).

Benjamin Garrigues on January 3, 2011 6:01 AM

this may be overly simplistic but could the magic dial google needs to turn simply be to adjust how the date something is published adds to the ranking?

on most searches say for "lady gaga" they should return the content with the most current date.

on a search for "binding a select list with MVC.Net" the scraped site is going to have a more recent date but should be ranked lower.

either way this problem seems to show up when using google to research technical solutions more so than other things.

ctrlShiftBryan on January 3, 2011 6:02 AM

I link to StackOverflow quite a bit in my Buzz. I wonder if Google has though of using Buzz as an input? :)

Peter Da Silva on January 3, 2011 6:07 AM

Google has been giving terrible results for product searches for a long time now. I tried to research a new dishwasher a few years ago and Google was a mess.

Google's main problem right now is that they think they can stay ahead of the black hat SEO with algorithms. Take it from somebody who used to work in the anti-spam and anti-virus world ... you can't! You need to use more than algorithms, in particular you need to use feedback from your users.

Google has a huge database of data on it's users. Google knows I've been using google and gmail for years, that I'm a real person, I use Google search dozens of times a day and I'm pretty technical. If Google had a "this is spam" or "this is not useful" button the would have millions of pre-validated curators to help them filter results.

Robert Osborne on January 3, 2011 6:22 AM

I've been seeing a ton of efreedom.com & questionhub.com data dumps of stackoverflow outranking you all through the holiday break.

Comforteagle on January 3, 2011 6:25 AM

I don't think "social" is the answer to better search results. In fact, I think it makes it worse.

- First of all, there is no reason to believe that spammers cannot manipulate a social search engine. Just look at sites like Digg.com where at one point people were getting paid to blindly vote content up.

- Even with educated, moral users minus the spammers social search ranking would most likely result in search result popularity indicators, which is not necessarily actually the best search result.

- It is questionable in the first place whether users at all would rank search engine results. They just want the result.

I'm hoping this recent problem can be solved by an algorithm, as I place more trust into that than mankind itself.

Fchristant on January 3, 2011 6:26 AM

The scrapers are probably doing lot of SEO optimization. It is time for stack overflow to hire some SEO services. Wikipedia is not monetizing in anyway other than donations whereas stackoverflow does display ads of its own so why not hire someone to do SEO and stay on top?

Nilesh Jethwa on January 3, 2011 6:43 AM

As I understood you're talking about scrapers, that try to cheat google algorithms (if the algos change, the cheating will eventually evolve) but you're blaming google, and not the scrapers??

What about SPAM, whose fault is it?

(just to be clear, I see google in this case more as a victim than as a villain)

Leonardo de Oliveira Martins on January 3, 2011 6:56 AM

I hate to play devil's advocate here, but I've noticed that efreedom.com, one of the SO scrapers, actually provides significantly better search and related question generation than SO does. There's a fine line between being a leach and being a value-added content aggregator.

Erikengbrecht.blogspot.com on January 3, 2011 7:07 AM

I've had a bad taste in my mouth for all things Google for a looong time.

At first there were some growing pains with it, but I've moved to Bing as a search engine and at this point, to me it feels speedy and lean like Google used to.

When I have gone to Google to look something up, I find that the result list is a near useless mess of ads, spam, and a general waste of time.

me.yahoo.com/a/wFcHGRpgnua_OqaxYt1WsXU_M_Lw1A-- on January 3, 2011 7:07 AM

As I understood you're talking about scrapers, that try to cheat google algorithms (if the algos change, the cheating will eventually evolve) but you're blaming google, and not the scrapers??

Google has always been about organizing oceans of chaos into something manageable and searchable. Lately they've failed in two respects. First, their searches are finding more noise and less signal. Google is supposed to be chock full of the best minds on the Internet and their algorithms are being beaten soundly. Not only has SO been s

Clintp on January 3, 2011 7:24 AM

(continued... damned mouse touchpad error)

...Not only has SO been squeezed out of the rankings, but I found Christmas shopping online to be much worse this year than last because what I wanted was buried under crap search results. Maybe they've stopped trying or caring about search.

Secondly, Google is the 800lb Gorilla of the Internet. If they wanted, they could simply crush anything that opposed them. I've been dying for a "never show me results from this site again" button in Google's search results. One click and the offensive scrapers go away.

Bing, on the other hand, seems active in tweaking and refining results...

Clintp on January 3, 2011 7:30 AM

>> I've been dying for a "never show me results from this site again" button in Google's search results.

Indeed. A content-based algorithmic search with the addition of user tools should be the way to go. I'd really like to manage my search results, and I don't mean in the way of voting for links like Google has implied sometime ago with their social searching services. That won't solve the problems and will introduce new ones (like social engineering or regional/cultural encroachment).

The current model is becoming expired and the "market" of content consumers is becoming less relevant in Google search results. This was once the great novelty of Google and what elevated them to their present status.

Mario Figueiredo on January 3, 2011 8:11 AM

Reminds me of Ben Croshaw's comments on user-created video game DLC: "...and don't tell me user ranking is the answer, because anything that references Naruto will automatically get five stars."

Chris Doherty on January 3, 2011 8:43 AM

@Clintp Interestingly that "never show me results from this site again" button used to be in google's results but isn't anymore. I miss it! Meanwhile we can try this extension: https://chrome.google.com/extensions/detail/ddgjlkmkllmpdhegaliddgplookikmjf

Scott Willeke on January 3, 2011 8:44 AM

I've noticed Google results being gamed for the last 6 months at least. Too many sites devoid of actual content being listed at or near the top of the results. software.informer.com was the first I noticed, but it's only gotten worse.

Lately I've switched to duckduckgo.com and msdn for any technical searches. MSDN even includes stackoverflow results!

Jobu on January 3, 2011 8:45 AM

This is an interesting twist on the software monoculture problem. By being the overwhelming favorite, Google gives spammers a single target to focus on.

On the other hand, unlike viruses, there are no inherent platform differences that prevent spammers from tweaking their content scrapers to poison other search engines as well. So if Bing were to gain more market share its results would likely start to drown in noise as well unless they have some secret sauce (or armies of content reviewers) to defend the walls against the barbarian hordes of spammers.

The problem with adding a button that says "this web site is useful" or "report abuse" is that it shifts the battle to gaming that metric instead.

The other problem is that from the algorithm's perspective, information is information, so who cares if some content scraper serves up information copied from someone else as long as you the searcher get the answer you're looking for, right? In some cases, the sites in question could be legitimate mirrors. When discussing flaws in the ranking algorithm, this goes to the heart of how you phrase the question -- assuming there are no flaws in the software, it is probably performing as intended, and this is essentially a garbage in/garbage out problem. Unfortunately, there is a lot of garbage on the Internet.

Peter Amstutz on January 3, 2011 8:46 AM

I do find that if I search for a product I end up getting drek back, but on the flip side it is drek offering to sell me the product, which is a reasonable assumption. The search for iphone cases mentioned above is a good case in point.

I will admit to being somewhat of a Google fanboy, but then I also have a lot of patience and am willing to venture as far as page 20 in search of useful results, I'm also making friends with Google shopping, not to mention continuing to read magazines. I find that helps me find products to search for.

Personally I've never had a problem with searching for technical data, but I understand that if you're producing it, that could be an issue due to the amount of drek and scraping that is served up.

I'm sure they'll get on top of it, otherwise people will vote with their feet so to speak.

John Doh on January 3, 2011 8:52 AM

It seems like search result quality went down with the recent real-time searching update. Maybe stuff like real-time twitter search beating Google made them rush out a non bulletproof real-time algorithm. Most likely this will improve with time as long as search is still the highest priority at Google.

Ryan Christensen on January 3, 2011 9:06 AM

Google is not gravity, it is a part (only a part) of the mechanics of natural selection (Darwin is The Dude, not Newton). The ecosystem has changed, which is all too predictable, but the law (force?) is still there. That something else will have to be added to the picture, possibly on top of Google - probably inside it too - is pretty obvious: we a still at the rehearsal phase of the Web. And Google still has a lot to show in the infrastructural front. Also, we need to think differently about our own data, and we will.

But I have to say, agreeing with you, that it is not the semantics, the secret is still on the syntax. Syntax is our stuff.

Aslemos on January 3, 2011 9:27 AM

Maybe it's time to use blekko?

Adam Rich on January 3, 2011 9:32 AM

Just thinking out loud, but Google did make some changes this year... adding Caffeine and all.

Ian Philpot on January 3, 2011 10:01 AM

I wholly agree as to the poor recent performance of Google's search results. I was just recently looking for binaural audio (I have my doubts, but was curious) and the results Google returned were garbage, with many results hosting identical content. The scammers are currently winning, or Google has failed to continue to implement its core concept.

That being said, in the search realm, this seems similar to what happened in the earlier days when Lycos promoted itself as having more indexed pages than any other search engine. That was a relatively easy hallmark to beat, so Lycos died rapidly. If someone else comes along with a much better search algorithm than Google at this stage, they just might have inertia like the early Google did. Or, more likely, the wealthy Bank of Google will buy them out.

Greg Webster on January 3, 2011 10:06 AM

Great post, and a great idea Iraê to add Delicious into the algorithm.

I agree that we should be using other search engines and also like comparison engines such as http://blindsearch.fejus.com/ but believe it will take a huge upheaval for what we have learned as a species over the last 10 years to change behaviour on the sort of scale needed to alter the statistics as above.

Google has fallen foul of it's own algorithmic success and is at a stage where diversification seems to be their strategy rather than adding in other methods for improving results. Personally I can't see Google winning and am keen to see what new players will come into this space. I believe more complex algorithms that search multiple resources and think/calculate longer before results are returned could be the way to go.

timaldiss on January 3, 2011 10:07 AM

While annoying when it happens, this is really nothing new. Several years back Google search results were inundated with parked domains only serving ads and other such useless pages. Google eventually cleaned up their algorithm. Sometimes their ranking system is gamed, other times they may make changes reducing the quality of rankings.

I considered switching to a new search engine and before I could find a better one, they fixed the problem.

Complain and then give them a chance to improve. It's a well established cycle that works very well for them.

Michael Silver on January 3, 2011 10:45 AM

I've been noticing for some time now that the results have not been as good as they were and I regularly find myself either trying more and more detailed terms or other search filters to help find what I'm looking for, and often it is not in the first 3-5 results.

Right now, my default engine is Bing and I'm finding better results there for some queries but they haven't indexed as much of the web as Google has so Bing's results are good, but still needs more time to develop.

Google on the other hand has been getting less and less reliable and I'm finding more and more content scrappers showing up in the top results and from what I can tell I've been noticing the decline in quality ever since the Mayday Update from last year. Since that time the quality of results for longer tail stuff have been a lot more inconsistent and there seems to be a lot more non-relevant data showing up in the longer tail searches.

Also, I don't like Google trying to show me results before I even typed in what I want. If I'm looking for a new car, and type in 'new cars' in Google, they end up giving me 'netflix' for the first letter n, 'netflix' again with the second letter added (ne), 'New York Times' when I type in the third letter (new), 'New York Times' when I add the space (new ), and if I add the first letter for the next work I still don't get what I'm looking for 'New Century Bus' for (new c), 'New Carrollton Metro Station' for the next letter (new ca), and it still doesn't get the search right even if I type in the last letter so now I've typed in 'new car' and the results that I'm getting in the instant are for 'New Carrollton' and noting about new cars is coming up. However, if I look at their suggestion list the actual result that I wanted is showing up as #3 on their list, but the problem is that I typed in the exact term I wanted to search for and instead of giving me what I wanted, I'm getting some other search that is not even close to being relevant or of any quality for me. With this in mind, I personally think their instant search is a joke, waist of time, and a big nuisance because I got 7 different search results for what I typed in and not one of them is actually relevant or what I'm looking for. What a big mistake Google made on releasing what I call their 'Crystal Ball' search where they are trying to predict what you want but are not doing a very good job of it.

SalSurra on January 3, 2011 11:12 AM

Great blog post. I can't imagine what effect an increase of that 88.2% will have if Google makes a better model.

Brian R. Bondy on January 3, 2011 11:24 AM

Jeff,
Do you continue to interact with Matt Cutts on this matter? Matt's opinion would be most interesting and most convincing.
You have not provided any material evidence any particular search, etc. to prove what you said in this article. Do you own any statistics? I admire StackOverflow, I know that you are a well know person in Web / programming circles, but it lately became very fashionable to attack Google.

Somewhat related article was published on TechCrunch (no real data either) at http://techcrunch.com/2011/01/01/why-we-desperately-need-a-new-and-better-google-2/

I'm pretty sure there will be related discussion in Google Buzz, hopefully with Google employees, including Matt Cutts at
http://goo.gl/6eVTw
http://goo.gl/xCDsD

Thanks

Vladimir Kelman on January 3, 2011 11:26 AM

Over the last year or so I've noticed Google's results getting worse. I basically taught myself design and front-end development by googling. Now, when I try to google for the most basic of searches I have a hard time finding the god content that was once right in front of me.

I've had to type more detailed searches, use the timeline features on the left sidebar and I started using delicious as a search engine for web related stuff more and more.

An article I read last night mentioned blekko.com (often times much easier to find what I'm looking for there) - I tried it out a few months ago but added it to my bookmarks bar recently and have been getting used to using it the last few days.

I'm glad other people are starting to notice google's bad results and hopefully the more people talk about it they'll the more google will work on improving and getting back to the search results of a year or two ago when I was actually able to find stuff...quickly.

I also noticed 'stack scrapers' recently and figured you guys would be a little upset. It's not their users creating the content, why the hell should they make money piggy backing off of you. But yes, if someone scrapes your content the original content should always be placed first in the results.

Good luck, hope you guys get it resolved and hope I get my google back.

hav0k on January 3, 2011 11:27 AM

I don't think we should be surprised by this. Google might say that they're in business to make the world a better place, but let's be honest... they have a responsibility to their stock holders to be as profitable as possible, and I believe that letting some of the scrapers move to the top of the list can only be padding the bottom line at Google. After all, how many of those scraper sites have placed AdSense ads on their sites?

AJ Rabe on January 3, 2011 11:29 AM

So, let's see:
1. Google's primary income comes from AdSense ads.
2. StackOverflow doesn't have AdSense ads.
3. efreedom have Google AdSense ads.

If you were Google, which site do you want people to go to?

You do the math.

InsomniacGeek on January 3, 2011 11:45 AM

Perfect example: I just searched "Android tablet" and clicked on the Google News tab. The 1st link I'm offered is for a site "TMCnet.com". It talks about Toshiba's new tablet being launched later this year. But throughout the article, it constantly links back to the "REAL" sources of the article, engadget and crunchgear.

So, how does an article by TMCnet.com, which is basically regurgitating what the other 2 more legitimate sources are saying, JUMP AHEAD of the actual sources? How is that possible? Interestingly, I didn't find any Google Adwords on their site, so that's not the motivation in this case.

RobertNaum on January 3, 2011 11:48 AM

Yes, converging feelings about social vs algorithmic search, I would say it is a recurrent rethoric now at each year's end.

Last year's passage (2009/2010) we had predictions and high praise of "real-time search" which could make Google bite the dust.

Where is real-time search now ? Where is real-time social search either ? the point of it and the results of it ?

thierryl on January 3, 2011 11:54 AM

I have long held that Search (ie, Google) would eventually bend to an anti-network-effect, where the SEO-gamers would eventually win and smaller search engines would flourish. I'm fond of DuckDuckGo right now. However, Google has gotten their edge back several times over the last few years, and I wouldn't count them out. I agree that Google is currently losing, and I recommend people try the blind search tests themselves.

I believe Google's strength will come in personalized search results, and I don't think Google is using personalization as much as they need to. If I bypass all the other links and go direct to StackOverflow every time, it would seem that - for me - this should work itself out quickly. I would be interested in whether your experiments with StackOverflow were with "clean accounts" or crusty accounts like mine, where the searcher is a known technologist.

Regarding someone who posted about categorization, Google's Caffine architecture, and the search results currently being returned, show a high amount of diversity. They're clearly categorizing and showing "best in category" in the front page results. This effect is positive for a number of kinds of search, but negative for "tight searches" (like iphone 4 covers, where you might end up with a ipad cover taking a slot due to a diversity algorithm, to the point where you're only delivering 4 or 5 results that were tight).

An improvement I would like to see in Google is a Google Labs experiment with a prominent "more like this" button. I'd rather do my first search and drill down, and it's clear Google has the categories and pre-calculated math to do so.

If I were out to game Google right now, I'd be building a very human-like browser (or using mechanical turk) to search for terms and click on my links. I suspect Google has greatly raised the priority of link-click in their reputation scheme, and gaming that system wouldn't be terribly hard. The benefit of blending personalization is hopefully I don't look like most mechanical turks.

Bbulkow on January 3, 2011 12:18 PM

@Konrad: "To stay in your metaphor, it’s clearly time for a paradigm shift, a kind of Einstein of web ranking algorithms."

That's not staying in any metaphor, bud. It's all over the place. =)

Tony on January 3, 2011 12:19 PM

I'd written about what is the intrinsic flaw of algorithmic search a while back, this may be of interest:

http://lesswrong.com/lw/28r/is_google_paperclipping_the_web_the_perils_of/

AlexanderM on January 3, 2011 1:10 PM

Not meant as an insult: BUT it's very very difficult for Google to decide if a site is a content farm, a ripoff or "valid" content. How should Google decide if a link at stackoverflow is a link that comes from an SEO idiot or valid? If a link on del.icio.us/digg/reddit is valid or simple SEO?

But I'd agree that Google should react faster. Especially ripoffs that do not confirm to the cc-license could be detected (at least mostly) automatically.

Ulrichvoss on January 3, 2011 1:13 PM

A social approach is the solution, but it MUST be designed so that it may not be gamed. The best way to do that is to allow me to vote up or down the search results and to allow me to blacklist/whitelist sites and to OPTIONALLY include my friends black and white lists.

It is this component of including friends, i.e. people I already trust, that ensures that it won't be gamed. If a friend of mine tries to game me, he/she won't be my friend for long. So it's self-policing.

Charles Scalfani on January 3, 2011 1:15 PM

Shoot me a DM next time you need a case, I'll send you a good one for free, save you from needless Googling and Amazoning! :) http://www.myGearStore.com

Benvanderbeek on January 3, 2011 1:17 PM

Google does a great job filtering spam in gmail. I'm not sure how important the "report spam" button is in this, but it is certainly somewhat satisfying to press it.

Where is the will to do the same thing for their search results?

I'd like to see a similar button in Chrome (for starters) for social rating of spammy websites. Other comments have noted that this would shift the goal posts to gaming the social rating system.

Maybe one solution to this would be to weight ratings by reputation. So Google detects that you are someone who rates spammy web sites highly, and devalues the rating you have applied to all other sites.

Jasonharrop on January 3, 2011 1:18 PM

You're right on except for this one statement:
"when was the last time you clicked through to a page that was nothing more than a legally copied, properly attributed Wikipedia entry encrusted in advertisements?"

On a growing number of search queries in Google, I'm seeing results from Ask.com that are Wikipedia articles with ads outranking the original Wikipedia posting.

Jakeludington on January 3, 2011 1:26 PM

I've been running into a flood of these scraper sites in my search results, and more than anything I just want to exclude them. I would like to click a link next to the result to exclude that site from future searches; there's no content I'm interested in on that site that shouldn't show up as a hit on the original source.

Providing that feature might solve the problem for two reasons: I don't see the scraper sites so my searches are more to my liking (and google works for me so I come back to it,) and also, google can use a large number of explicit "exclusions" to affect the rankings. They could treat it as feedback, equivalent to users saying "this site is not relevant."

Jim Rogers on January 3, 2011 1:51 PM

I actually have been hoping for a change in Google for a while. While Blekko shows promise, it isn't exactly what I was hoping for. And, though I know some of the following are a bit of a stretch now, they will be invaluable in the future.

First, I want to be able to filter results from my search. (This is the opposite of what Bbulkow suggested), but his option would be good too) I want to be able to click something which says, "This site is bogus and should not be in this result set" or "That has nothing to do with what I am looking for". When I look for a legitimate answer for a question, I want to be able to tell Google to take about.com and shove it.
I want to be able to search for symbols. I mean seriously, if I'm trying to find an email, why does it need to be changed from "foo@bar.com" to "foo bar com" (I'm a bit sensitive here, my last name is Allen-Poole).
True Boolean logic. I want to look for ((this and that) or (that and another)) and not (some-other-thing).
I want a means to search for linguistic constructs. For example, if I am looking up John Smith, I want to have a search which looks for the name (two words in close proximity, separated by a middle name or a middle initial). This is more than possible.
I want regexp. That is just insane though. I don't expect to grep the web any time in the near (or maybe even distant) future.

And what am I willing to trade? Time. I remember the 90's. I remember preferring AltaVista because its results were just slightly better and its logic seemed more reliable. But the amount of time I will save in proper results is invaluable and worth far more than whatever extra seconds that it takes crunching the numbers on their end (even extra minutes!).

Just think about this: it takes at least a second to read the title of a google link. It will take another couple of seconds to evaluate the text beneath. It is also not unreasonable for a website to take 3-5 seconds to load completely (though this can be optimized with tabbed browsing (though that can also decrease as the full will not appear)). It takes an additional 3 seconds (minimum) for a human to read and parse a site, realize that this is not what you wanted, and then move the hand back to the mouse.Now, if you were to have a questionable search (say, the dishwasher ratings example) and the first result is bad, the second and third result are maybes, the forth result is Amazon, and the fifth result is one which is relatively useful. This means that you will waste at least 5 * link-text + 4 * subtext + 2 * site-viewing to get to the result (assuming you stay on the good result). That makes a minimum of 19 seconds of completely wasted time before getting to something truly useful. In this case probably more because you likely will stay on the mediocre results for a while longer than 3 seconds.If Google were to give us these options, if it were to make our searches better, even if it were at 100% increase in search time, we would end up with a net benefit (I've not had too many searches take 10 seconds recently). The first point alone could net some extreme benefits and it reminds me of (http://en.wikipedia.org/wiki/Travelling_salesman_problem#Ant_colony_optimization) a solution to the travelling salesman problem. And, while this is still something which advertisers could use to our disadvantage, it would be a lot harder for them to do so, especially if these were implemented on a per-user basis.Now, I know that I am a lowly voice in a sea of spam, but seriously. Google has the ability to implement this. I've read their specs and I think that, if they wanted, they could even make a way to grep the web. For the first task, it wouldn't even need to involve stored data -- it could all be tracked within one session. The next question is whether Google will care.Amusingly, I feel it obligatory to add a link to http://allen-poole.com so that some day Google may look upon me and smile.Cwallenpoole.wordpress.com on January 3, 2011 1:52 PM The only feature I need Google to implement right now is giving me the ability to blacklist sites in all my queries. I've long wished that I could blacklist experts-exchange, and with the proliferation of scraping sites over the last year, that desire has become even greater.You could possibly make it social (my "friends" blacklist can be added to my own), but don't use blacklists to influence rankings. And no, don't do any peer votings for ranks either, as these will lead to more abuse and just be added to the list of SEO techniques.This can't be that hard. I can already add "-site:experts-exchange.com" to my queries to remove the sites. Why can't it be an option in my google account setting to add that to all my searches.Evan Morgoch on January 3, 2011 1:55 PM I find it hard to believe that this isn't intentional by Google, although I imagine the attention gathered by this article will change things dramatically.Davidsimbroglio on January 3, 2011 1:57 PM Btw, there is an extension button for Google Chrome to report spam. It automates a part of filling in their report form. HenkPoley on January 3, 2011 2:10 PM Its not you dude. Google is becoming the new Yahoo, one spam result at a time.The sad part is without google the web is nothing. With all the technology improvements, nothing has really improved.Its time to get VC out of tech and start building things that work. The web has turned into a get rich quick scheme.Rendion on January 3, 2011 2:47 PM Hi Jeff, I passed on the examples that you sent back in December and the team is actively looking at improvements and changes they can make based on that feedback--thanks for sending it.I was curious about the link to "Google, Google, Why Hast Thou Forsaken the Manolo?" and so I checked that one out. It's true that our algorithms don't currently think that's a great site, so I looked into it more. The disclaimer says "Manolo the Shoeblogger is not Mr. Manolo Blahnik." It's a *different* Manolo in the shoe industry.So I picked a url, let's say http://basement.shoeblogs.com/category/bedding/ . Pretty much every post looked like "buy this type of bedding," usually with an affiliate link. And over on the right-hand side are links like "Shop hassle free and buy unique Duvet Covers at thecompanystore.com" that look an awful lot to us like paid links that pass PageRank.I support the right of this blogger to put whatever they want on their domain, but I also support Google's right to decide how to rank our search results, and I don't think we should be obligated to rank that site highly.I appreciated the rest of your post and it's safe to say that people inside Google are discussing it and how we can do better.Matt Cutts on January 3, 2011 3:03 PM I'm no expert but what about taking the new syndication-source and original-source meta tags a step further.Original content can be pinged, timestamped etc. w these tags. Webmaster tools could be used to report sites that are outranking the original content. Database would verify and adjust rankings.Rewrites etc. would still happen but should help clean things up a bit in addition to giving content producers (and Google) an easier way of dealing with this problem.Kevin_Szprychel on January 3, 2011 3:41 PM This looks pretty inevitable.Two aspects come to mind:1) It's an algorithm not human thought that's at work here. That gives an arms-race, the dark-part-of-SEO will catch up even if they started out pretty dumb.2) Google makes it's money from adverts. A site that has a lot of adverts is working for Google. I really can't imagine them stamping on such sites like they're bugs. For me some of them are just that bugs, so we have a disconnect! In the absence of a published algorithm (even if it needs updating daily or more often) this sort of suspicion can't be resolved.Human judged content (DMOZ anyone!) looks like an answer. Many times I look at what "social" delivers I shudder. A great average of everybody, it seems to me, is not the answer.Maybe the web just needs to fracture. Personal control over how your own search works, sharing data with people who's opinions you respect, sites that work your way, less rubbish, less time waste, more productivity.We could end up with different worlds, as sketched in some SciFi books for a long time. Those who live on the web, consuming, following, never creating. Those who disconnect, think for themselves enough that they deliver new and valuable work.The web has altered our lives. It's time those who care get back into the loop. Control your web so that your life is yours, not a side-effect of a cacaphony of "important" web companies.Mike Gale on January 3, 2011 4:03 PM Seems to me that relying purely on content for indexing isn't going to work any more. Each web site comes from a hierarchical division of address blocks. The existence of a "bad" web site within a given address block can and should impair the score of every other web site within that address block, to a lesser degree as we ascend the address block hierarchy. The same concept should be applied to registrars.In other words, if my ISP hosts a lot of spam sites or there are a lot of them in my address block, my site is going to take a penalty, regardless of its content. I therefore have an incentive to seek out a reputable ISP, and reputable ISPs have a very solid reason to push out spam sites.Eradicating this trash means making it harder and harder for it to find a "home". I can't think of a better way to do that than to have ISPs actively working on the problem, to retain their wider customer base. If they don't have a wider customer base, and it's all spam? Page ranks from that ISP will snuggle up to each other at the bottom of the pit.The basic problem here is that its too hard to keep adapting like crazy to all the ways of restructuring content, times all the possible web sites. The number of ISPs and address blocks is, however, entirely tractable for this kind of problem.Of course, this can punish entirely innocent web sites, until the system as a whole shakes itself out. It would be nice to have this particular omelette be break-free, but I don't see how to do that.Ross Judson on January 3, 2011 4:12 PM I use Bing at work and Google at home (don't ask) and as odd as it is, I do get better results with Bing.Craig Deubler on January 3, 2011 4:21 PM There was an algorithmic thing back in 2006 that included TrustRank. While it wasn't exactly a social recommendation type of thing, it did distribute GoogleJuice based on links from trusted sources.http://weblogs.asp.net/jgalloway/archive/2006/01/11/435076.aspxIf there's an element of TrustRank in the current algorithms, it seems like that probably needs both a reset and a higher weighting.Jongalloway on January 3, 2011 5:06 PM @Matt CuttsI can see you took the time to read, analyze and post a comment. That's very decent of you. Unfortunately I can also see you only addressed Jeff, ignored any comments from commenters in here and approached the matter purely as a ranking issue.Since Google Search is meant to be a service to the "user who searches" and not a service to the "user who publishes", I'm unsatisfied by your comment. But not surprised.Mario Figueiredo on January 3, 2011 5:18 PM The whole PageRank conundrum reminds me of the parable in Gödel Escher Bach about the phonograph that breaks when you play a specific well designed record. GEB was referring to incompleteness but it's an equally good metaphor for computer security and quality-algorithms like PageRank.If there's sufficient motivation to find your algorithm's weak points and exploit them, it's going to happen. Complicated algorithms just require more complicated and better designed inputs. Justin Scheiner on January 3, 2011 5:19 PM Hi Mario, it's actually my 11 year anniversary this week. I'm out of town with my wife, so I only have limited time to slip away and post responses. Suffice it to say that plenty of people in Google have read this article and the other articles Jeff mentioned, and lots of people will be discussing what we need to do next to improve things.Matt Cutts on January 3, 2011 5:28 PM @Matt CuttsYes, Manolo the Shoeblogger's site is like a lot of fashion blogs, in that it has decent number of affiliate links. However, you didn't answer the central question posed by the Manolo in the post you've referenced, "why are the scrapers ranking higher than the original content?"You, and others at Google, have harped on for years about the need to produce interesting and original content, and yet, if a site which produces plenty of original content doesn't throw exactly the right levers in Google's Rube Goldberg system, you'll preference a dozen content scrapers over it. Manolo is very well known among fashion people and in the fashion press, in fact he pretty much invented the Fashion Blog...http://en.wikipedia.org/wiki/Fashion_blog#Early_fashion_blogs So, again, why should the content scrapers who are stealing his work be ranked higher than he is?Del Davis on January 3, 2011 5:54 PM Lets hope they do and things do get improved, Matt. There's been a growing disconnect between Google Search and its users for the past... couple of years, I'd say. To the point that previously very rare statements like "Google search engine isn't good anymore" are becoming more prevalent. Something that would be unthinkable before.Being that this is also the period in which Google introduced the most relevant new features and changes to the search engine UI since its inception, maybe it's time (and excuse me the bluntness) Google realizes that may not be what users actually require the most.I'm prepared to accept also we are simply a non representative minority. But I do seem to witness a growing cry of protest. With alternative search engines taking their place in the market offering competitive possibilities, all care is not enough. Remember how Google itself rose.And my congratulations, BTW!Mario Figueiredo on January 3, 2011 5:57 PM Like it was said way upthread... I see this as a manifestation of the Windows/Mac malware thing. All the bad guys are optimizing their dark SEO for Google, not Bing. If they decided to focus on Bing, given time to catch up to their extensive knowledge of Google internals, the same would happen to our bingy buddy. A thought: does Google factor in domain registration time to its algorithm? This seems like a reasonably accurate heuristic for tracking original content vs. scrapers. Obviously a "reused" scraper domain would be the problem.0xabad1dea on January 3, 2011 7:01 PM I remember seeing this auction on flippa a while back: https://flippa.com/auctions/102189/1-iPhone-case-site-11kmonth-profits--2-million--pageviewsmonthThats the number one search result for "iphone 4 case" above apple, above amazon...crazy. Their auction description gives more detail on their seo efforts.I troll AM forums like wickedfire where I often find insightful threads like overstock dominating the SERPs for very generic keywords. Why is the #1 result for "watches" "luggage" "crib sets" Overstock? arent there more deserving and relevant results? is google and overstock profit sharing? http://www.wickedfire.com/shooting-shit/111865-fuck-overstock.htmlBryan on January 3, 2011 7:04 PM As a web developer and SEO enthusiast, I've been increasingly surprised at how hard it is to find anything on Google anymore. This holiday season was particularly frustrating. After three or four attempts to find a Tiffany's bracelet, I gave up and went over to Bing, where I actually found several pages of relevant content to choose from. One of my professors in college told about some theory (can't remember the name right now) where as you try and narrow down your hypothesis to get more and more specific, at some point, you actually become less and less effective and what you're trying to achieve. He used the visualization of an hourglass. As you narrow your results, you get to a finite point (the actual apex where the sand drops into the next chamber), after which point you get further away from what you're trying to achieve.To me, this is where Google is right now. They're trying WAY too hard to continue to generate revenue while delivering the most personalized, specific results to the user. People are catching on and their gaming the system, without penalty. The fact is Google is broken and it will take a while before it's fixed. Crash Override on January 3, 2011 7:25 PM Well, my blog / site is a lot smaller than yours, and I have put zero effort so far into SEO, but I get very little traffic from search. Most of my traffic comes from Twitter, Hacker News and DZone, with DZone being the biggest contributor.In any event, I do think you're right about the content farms and other "spam" clogging Google. It seems we need some kind of reverse Turing Test. A *person* can easily tell a chatbot from a human conversation partner, but can a few cubic miles of MapReduce engines?Well, unless you count sports reporting, that is. ;-)http://borasky-research.net/2010/12/30/sure-why-not-five-predictions-for-2011/Znmeb on January 3, 2011 11:00 PM Really useful article,, thanks google chrome..Indian Sareessarees on January 3, 2011 11:17 PM I have been disturbed by Google for sometime and in actual fact never use it as a search engine. From a hidden program installer on my computer to the new Google chrome which outright says its keeping all and I mean all your information on a cloud. It wont even let you download programs. I also noticed recently that Microsoft gave the makers of a game information with regards to how many players were playing the game for the month as well as how long they played it. Seems to me every iota of privacy is disappearing. Quite frankly it scares me silly that people continue to believe it will do no harm. What if the USA Government or some other controlling force demand the info be handed over? The same with the new Apple patent with the application they intend putting in your phones? There is really only one person you can trust with your personal information and that is yourself. I am not a scare monger but I have seen what harm a dictator can do. The only way to keep yourself and your information safe is to totally keep it off the net.Nicki on January 4, 2011 12:02 AM When I have a programming problem, I google for '{error message} {platform}', get the useless results, and then google for '{error message} {platform} stackoverflow' and i get a lot of good SO results and nothing else. In that respect, the system works, but yeah - you guys are missing out on a hell of a lot of good traffic.John Senner on January 4, 2011 12:10 AM Internet would be a much better place if we finally deploy some cryptography based ranking/kudos solution. I don't know -- like http://www.bitcoin.org/ but for rankings. Something that would made SEO impossible at all.dpc on January 4, 2011 1:09 AM Social search will work until the minute it wont work anymore. It' won't take long before spammers and scrapers find a way to beat the system like they managed to beat algoritmic search engines.It's allow more easy to beat a social sistem than a algorithmic one. I'm just saying, humans like to see the dancing bunies.Pop Catalin on January 4, 2011 2:00 AM +1 to personal website blacklists: I've felt a need for them for years. If I take the time to click a link, and understand that a website is somewhere I never wish to come again, I would like to be able to leverage that investment.+1 also to being able to "follow" other people's blacklists. It would be very dangerous, though, to make the stats about following public: it would probably create a gravity effect towards the most followed blacklisters, who could then become too powerful - and be tempted to monetize that power, as it happens on many social networks. This should be of interest to Google - it would also allow them to have better social graph data.If enough "trust communities" will grow, this should also lower the incentive to game the system, making SEO-only websites less lucrative.Talking about the scrappers, there are two very different situations here. The first one is where the content is Creative Common (or similar) and legitimately reproduced. In this case, there is no reason why Google should automatically give the first publisher a better ranking: if some of the scrappers published it in a "better" (whatever the metrics) way for the searcher, why shouldn't it get a higher placement? It would be very nice, though, if Google aggregated the similar pages, like it does in news: it is very annoying when you click several copycat links in a search.The other case is when someone steals the copyrighted content. This should definitely be penalized, and in theory it should be the law to do so. Considering the reality of things, though, it would also be very much in the interest of Google to help the original content producers protect themselves - giving them an incentive to produce even more good content. Considering how fast, and how often they crawl the web, Google could very often find who really published first, and if there were a meta tag about the copyright and license, it could at least warn the original publisher, if not find a way to penalize the stealer.Daniele Mazzini on January 4, 2011 2:39 AM I think what this shows is that over the long term, Search as we know it is broken. The only way out will be "hybrid curation" -- basically back to the Yahoo model at the highest level -- with algorithmic results (ala Google) into the depths of the curated high level web sites.Amazon is a great place to research products even if you don't shop there because there is curation and great reviews.[dc]Dave Chapin on January 4, 2011 3:32 AM well Matt C did say thatgoogle had taken resource away from some aspects of antispam and that would be returing this year.Hauntingthunder.wordpress.com on January 4, 2011 3:54 AM One of the biggest flaws of Google is that it gives way too much importance to domain names. If you search for 'iphone 4 case' you see plenty of websites like iphone4case.com, getiphonecase.com, iphone4gcasereview.com, www.iphone-4g-case.net, www.4iphonecases.com etc. I don't understand why domain name is such an important criteria for ranking search results. That needs an immediate fix.Vasuadiga on January 4, 2011 4:27 AM Interesting article. My responses to a couple of the comments:First: "[...] where the content is Creative Common (or similar) and legitimately reproduced. In this case, there is no reason why Google should automatically give the first publisher a better ranking: if some of the scrappers published it in a "better" (whatever the metrics) way for the searcher, why shouldn't it get a higher placement?"As a consumer I would rather reward (with traffic) the content creators for making knowledge available to mankind than I would the scrapers who have not generated anything new. By doing this, I assume, I am encouraging them to continue to create content - looking at the scrapers instead is less likely to have that effect. So, the scrapers are by definition not "better" and if the metrics think they are then the metrics are broken.Second point: the categorisation into "social" and "algorithmic" search seems to me terminologically inexact when a key element of the "algorithmic" search is which sites have incoming links from other people. If those links are put there by people, that's a pretty social algorithm ;-) Perhaps the distinction would better be drawn between "anonymous" and "personal social" search. Coruskate on January 4, 2011 5:23 AM @VasuadigaWell, it really isn't the domain name that is influencing the results. Along with that domain there's a legion of SEO techniques that are the actual responsible for the website placement. There never was, and still there isn't, any reason to believe the domain name factors in a website rank. Neither it would make any sense. What happens instead is that a domain name like iphone4case.com facilitates the creation of a link anchor text that may be more relevant to Google's algorithms (it is believed that a link anchor text is important).So with a domain like that, the owner is effectively creating a commercial name that goes like "iPhone 4 Case". Contrast that with the same business, had it been named mobileshell.com. When someone links to their business, the link anchor text and surrounding text could read as:- Find your iPhone cases at [u]iPhone 4 Case[/u] - Find your iPhone cases at [u]Mobile Shell[/u]On the first case, both commercial name and anchor text accurately reflect the business, whereas the more creative second option will however produce an anchor text that doesn't. So when searching for the company name, "Mobile Shell" may produce a lot of false positives with links to the military or engineering areas, whereas "iPhone 4 Cases" will not. On the other hand, when searching for the more generic term "iPhone cases", the first company is at an advantage because there's a real chance that the vast majority of anchor text that link to their website include these exact terms (the plural form is largely ignored by google).Mario Figueiredo on January 4, 2011 5:26 AM Have you tried Googling "sugar bowl" lately? Very misleading first listing.StarTrekRedneck on January 4, 2011 6:15 AM How does Bing fare in all this?? Is Bing equally scraper-infested??Should and could Google and Bing et al. create scraper blacklists similar to anti-spam blacklists??I have not yet experienced this scraping etc. -- are such issues related to how general or specific are one's search terms??Thank you, TomTom Lyczko on January 4, 2011 6:23 AM Crowdsourcing is the answer, IMO. If there's one thing Google has, it's a lot of users. Whatever happened to SearchWiki (http://googleblog.blogspot.com/2008/11/searchwiki-make-search-your-own.html)? It had a pleasant user interface, well-integrated with the results. I feel that such a mechanism, with a reputation system of some sort (perhaps subscribing to weighting results as edited by trusted groups of users) could drastically improve search result quality.Goran Zec on January 4, 2011 6:50 AM From google perspective : How about letting users vote to bury sites that just copy content?From web browsers perspective: How about making an plugin to preprocess google result filtering out sites on a black list?From a user perspective: Use alternatives to google (yahoo, bing, etc), the less people use google, the more google is forced to improve. Google replaced yahoo, but it can be replaced if they don't hear their users. Ricardo Santos on January 4, 2011 6:58 AM @Scott Willeke, Thanks! I installed that blacklist plugin. I've been wanting such an extension for some time: https://chrome.google.com/extensions/detail/ddgjlkmkllmpdhegaliddgplookikmjfI want to acknowledge Google as an innovative company that almost single-handedly made the world wide web useful. As of last year they'd crawled over 1 trillion unique URLs, an astounding amount of noise to sift through. I admire their engineering ethos and feel their business largely adheres to "don't be evil". That said, there is a real, serious problem with result quality. Google is a victim of its own success. The ecosystem they created is so profitable that it requires Google to spend inordinate time (possibly 50% of engineering?) keeping webmasters honest. Pick your metaphor -- traders gaming the stock market or bacteria growing antibiotic resistant -- bad websites are out-evolving Google.Ranking knowledge has become ubiquitous, and sadly knowledge of gaming an engine has become more important to content sites that writing valid, expert content. It's not just the spammers, malware sites, and scraper sites writing worthless keyword stuffed content and buying links. Google also made a deal with the McContent devil, Demand Media: http://techcrunch.com/2009/12/13/the-end-of-hand-crafted-content/Demand buys up search queries and pays writers a paltry sum (dollars) to write poorly-researched content on subject areas they often have little to no experience in. Demand makes a few ad dollars per article, with traffic exclusively driven by search (I've never met anyone who goes directly to eHow.com to browse.) In turn, Google takes a cut of Adwords dollars. In the short run, Google's bottom line looks better, especially on a Youtube site they've had trouble monetizing. Demand runs eHow, but you'll equally vapid content on Q/A sites Wikia, Yahoo Answers. Google needs to respond or their flagship search will suffer. The solution will be complex and multifaceted. In addition to small, incremental changes, I think Google will need to make some seismic ones. Google will face cries of injustice from "content producers" in the gray areas, but they need to stand tough. I run SEO program at a large US media organization, NPR. From the beginning, we've stayed above board - fixing coding issues, worked on syndication, and trained our writers on the very basics. We write first for humans. That ensures that Google crawls us adequately, but we do lose traffic to sites that out-SEO us, legitimately or otherwise. My long view is that this current state of search is not sustainable, and any efforts we spend beyond the basics are at the expense of other products we can build.It's easy for my organization to take this tack, however, because we're a well known brand and we can focus on other channels, such as social media and viral sites. Content producers should think about the tradeoffs they make when going broke for SEO -- it's impossible to quantify the traffic you don't get from Facebook/Twitter when you water down your content. Javaun on January 4, 2011 9:11 AM More comments» <a href="http://www.codinghorror.com/blog/2011/01/trouble-in-the-house-of-google.html" target="_blank" rel="nofollow">View the original article here</a><div class="blogger-post-footer"></div> </article> <article> <h1>Stack Overflow (the company) is now Stack Exchange</h1> 2011-03-21T16:58:00.005+05:00 The snack room at Stack Exchange got a wee upgrade today:Find out why (and read to the end to find out how to get your own StackExchange sticker) at the Stack Overflow Blog. <BLOCKQUOTE class=textmessage> Have you been wondering about Distributed Version Control? It has been a huge productivity boon for us, so I wrote Hg Init, a Mercurial tutorial—check it out! </BLOCKQUOTE>You’re reading Joel on Software, stuffed with years and years of completely raving mad articles about software development, managing software teams, designing user interfaces, running successful software companies, and rubber duckies. I’m Joel Spolsky, co-founder of Fog Creek Software, a New York company that proves that you can treat programmers well and still be highly profitable. Programmers get private offices, free lunch, and work 40 hours a week. Customers only pay for software if they’re delighted. We make FogBugz, an enlightened bug tracker designed to help great teams develop brilliant software, Kiln, which simplifies source control and code review, and Fog Creek Copilot, which makes remote desktop control easy. I’m also the co-founder of Stack Overflow. <a href="http://www.joelonsoftware.com/items/2011/03/09.html" target="_blank" rel="nofollow">View the original article here</a><div class="blogger-post-footer"></div> </article> <article> <h1>Stack Exchange for Jewish Life and Learning</h1> 2011-03-21T16:58:00.004+05:00 It seems like Stack Exchange is the perfect platform for questions about Jewish observance. After all, most of the Talmud reads just like Stack Overflow: a question, followed by multiple answers, usually with the highest ranking answer appearing first. The number of questions is infinite. If you would be interested in participating in such a thing, please commit to the proposal today. <BLOCKQUOTE class=textmessage> Have you been wondering about Distributed Version Control? It has been a huge productivity boon for us, so I wrote Hg Init, a Mercurial tutorial—check it out! </BLOCKQUOTE>You’re reading Joel on Software, stuffed with years and years of completely raving mad articles about software development, managing software teams, designing user interfaces, running successful software companies, and rubber duckies. I’m Joel Spolsky, co-founder of Fog Creek Software, a New York company that proves that you can treat programmers well and still be highly profitable. Programmers get private offices, free lunch, and work 40 hours a week. Customers only pay for software if they’re delighted. We make FogBugz, enlightened project management software for bug tracking, Kiln, which provides distributed version control and code reviews, and Fog Creek Copilot, which makes remote desktop support easy. I’m also the co-founder of Stack Overflow. <a href="http://www.joelonsoftware.com/items/2010/12/09.html" target="_blank" rel="nofollow">View the original article here</a><div class="blogger-post-footer"></div> </article> <article> <h1>The Wikipedia of Long Tail Programming Questions</h1> 2011-03-21T16:58:00.003+05:00 “Have you ever noticed how certain questions come up again and again on Stack Overflow sites?”— From The Wikipedia of Long Tail Programming Questions, over on the Stack Overflow blog. <BLOCKQUOTE class=textmessage> Have you been wondering about Distributed Version Control? It has been a huge productivity boon for us, so I wrote Hg Init, a Mercurial tutorial—check it out! </BLOCKQUOTE>You’re reading Joel on Software, stuffed with years and years of completely raving mad articles about software development, managing software teams, designing user interfaces, running successful software companies, and rubber duckies. I’m Joel Spolsky, co-founder of Fog Creek Software, a New York company that proves that you can treat programmers well and still be highly profitable. Programmers get private offices, free lunch, and work 40 hours a week. Customers only pay for software if they’re delighted. We make FogBugz, an enlightened bug tracker designed to help great teams develop brilliant software, Kiln, which simplifies source control and code review, and Fog Creek Copilot, which makes remote desktop control easy. I’m also the co-founder of Stack Overflow. <a href="http://www.joelonsoftware.com/items/2011/01/05.html" target="_blank" rel="nofollow">View the original article here</a><div class="blogger-post-footer"></div> </article> <article> <h1>Careers 2.0 (by Stack Overflow)</h1> 2011-03-21T16:58:00.002+05:00 One day, you’ll be telling your grandchildren about getting a programming job, version 1.0. You would send a “resume” to a “recruiter.” It included all kinds of silly information required by the esoteric resume ritual (foreign languages spoken, whether or not you play ultimate Frisbee, Microsoft-veteran status). This so-called “information” was utterly useless at determining whether you could program or not, but if you spelled everything right and used suitable fonts, you could come in for a day of interviews at which you would be asked to perform mundane programming tasks on a whiteboard.Careers 2.0 is here! <BLOCKQUOTE class=textmessage> Have you been wondering about Distributed Version Control? It has been a huge productivity boon for us, so I wrote Hg Init, a Mercurial tutorial—check it out! </BLOCKQUOTE>You’re reading Joel on Software, stuffed with years and years of completely raving mad articles about software development, managing software teams, designing user interfaces, running successful software companies, and rubber duckies. I’m Joel Spolsky, co-founder of Fog Creek Software, a New York company that proves that you can treat programmers well and still be highly profitable. Programmers get private offices, free lunch, and work 40 hours a week. Customers only pay for software if they’re delighted. We make FogBugz, an enlightened bug tracking and software development tool, Kiln, a distributed source control system that will blow your socks off if you’re stuck on Subversion, and Fog Creek Copilot, which makes remote desktop access easy. I’m also the co-founder of Stack Overflow. <a href="http://www.joelonsoftware.com/items/2011/02/23.html" target="_blank" rel="nofollow">View the original article here</a><div class="blogger-post-footer"></div> </article> <article> <h1>FogBugz/Kiln Demo</h1> 2011-03-21T16:58:00.001+05:00 If you weren’t able to make it to the FogBugz/Kiln world tour, a video of my presentation is up now on YouTube.(If you have a high bandwidth connection, try the “720p” option, which shows the screen more clearly.) <BLOCKQUOTE class=textmessage> Have you been wondering about Distributed Version Control? It has been a huge productivity boon for us, so I wrote Hg Init, a Mercurial tutorial—check it out! </BLOCKQUOTE>You’re reading Joel on Software, stuffed with years and years of completely raving mad articles about software development, managing software teams, designing user interfaces, running successful software companies, and rubber duckies. I’m Joel Spolsky, co-founder of Fog Creek Software, a New York company that proves that you can treat programmers well and still be highly profitable. Programmers get private offices, free lunch, and work 40 hours a week. Customers only pay for software if they’re delighted. We make FogBugz, enlightened project management software for bug tracking, Kiln, which provides distributed version control and code reviews, and Fog Creek Copilot, which makes remote desktop support easy. I’m also the co-founder of Stack Overflow. <a href="http://www.joelonsoftware.com/items/2011/02/03.html" target="_blank" rel="nofollow">View the original article here</a><div class="blogger-post-footer"></div> </article> <article> <h1>Stack Overflow 2010 recap</h1> 2011-03-21T16:58:00.000+05:00 (reposted from the Stack Overflow blog)2010 was an absolutely amazing year here at Stack Overflow. We grew from 7 million visitors to over 16 million, putting us in Quantcast’s top 400. We raised $6 million in venture capital, and we went from three full time employees to 27. We built a 7500 square foot office in New York, and we launched a ton of new features and sites, like Stack Exchange, a network of 33 Q&A sites on diverse topics from cooking to computer science. Stack Exchange grew 51% in December alone. Wow.The expert Q&A model that Stack Overflow pioneered is really working. The statistic I’m proudest of is the percentage of questions that get a good answer, over 80% (and many of the new Stack Exchange sites have 100% answer rates!)The true measure of success for any Internet company is how often people come up to me in swank hotel lobbies and offer to buy me meals, let me use their corporate jet, etc. But since there is a great deal of disagreement as to how to measure that, we track a reasonable proxy called “eyeballs,” on the theory that if a site is useful, people will load it up in their browsers and eyeball it.Traffic to Stack Overflow grew 131% in 2010, to 16.6 million global monthly uniques. *Uniques* are counted by cookies, so the number of human beings is less. We also measure the number of page views (top level pages loaded, which doesn’t count images and supporting files), which has similarly grown from 31.8 million per month to 72.8 million per month, i.e. 129% growth.Based on the number of people who do come up to us in hotel lobbies, we’re pretty sure that ALL the programmers in the world use Stack Overflow. (Source: completely made up. But seriously, when was the last time you met a programmer who didn’t use “El Stack”?) In order to keep growing and making the Internet more awesome, we have to expand into new subject areas, like Molecular Biology and Harley Davidson Belt Buckles. That’s what Stack Exchange is all about. Stack Exchange growth is insane. In six short months, we’ve gone from zero to 1.5 million monthly visitors, growing 51% in December.If, as planned, we continue growing at 51% a month, we will be bigger than Facebook in 15 months. We’re ALREADY bigger than ocn.ne.jp (No, I’ve never heard of that either. But we’re bigger). Jeff and I are already planning who will play us in the Aaron Sorkin movie. (Tyler Labine and Zac Efron, obviously.)Now, obviously, all this TRAFFIC isn’t worth a thing if people aren’t getting answers to their questions. That’s why our favorite thing to measure is “percent of questions answered.” And not just any answer will do, either: to count a question as “answered”, either the original poster has to accept the answer, or a third party has to upvote the answer. This is where Stack Overflow really shines compared to other Q&A sites: we actually get questions answered. Three of our sites actually have 100% answer rates!Last summer, we relaunched Stack Exchange as a democratically-driven network of sites on topics chosen by our users. Some of these sites are directly related to programming (for example, Game Development), but some are quite far afield, from English Language toCooking.We call it the Stack Exchange network, and at StackExchange.com you’ll find a directory of all of them, along with some hot questions, statistics, leaderboards, and other tools so that you can follow the sites and tags that you’re interested in.We learned a long time ago that the only way to get questions answered promptly is to have a critical mass of knowledgeable users, so we have an onerous process called Area 51 where sites are proposed, discussed, and voted on. If a proposed site doesn’t have critical mass, we just won’t create it. Even if it does get created, it has to maintain a certain level of traffic and quality or we’ll close it down.So far, 13 sites have gone all the way through the Area 51 process and launched. Dozens more are already in beta. Hundreds more are in active discussion and will launch when they reach a critical mass of interested participants.The development team has been knocking out new features at a constant pace. They built an amazing web-based chat system, and we’ve added literally hundreds of new features and improvements to the core Stack Overflow engine which we roll out continuously.At the beginning of the year, Stack Overflow LLC was just three developers working from home. In the spring, we raised $6 million in venture capital from Union Square Ventures and a long list of celebrity angel investors, which allowed us to expand rapidly. We hired a team of great people, including several of the high-reputation users that you know from Stack Overflow.We now have community managers, a sales team, two full time system administrators, and Very Important Administrative Overhead like myself, but most importantly, we have a great team of developers, in New York and around the world, building the next generation of cool features, like the important “wheel of blame” feature, which we can run at any time to calculate precisely who is responsible for anything that went wrong. (Contrary to popular belief, it’s not always Jason Punyon.)To make room for all these people, or, at least, those who live in New York, we rented a 7500 square foot, class A, super-elite batcave in New York and then fixed it up to be nice, with cool furniture including Aeron chairs and height-adjustable desks, and lots of glass to bring views and daylight deep into the batcave. And of course, we have private offices with a half dozen gigantic 453-inch monitors for each developer. And there’s an amazingly cool Star Trek couch. Does your company have a Star Trek Couch? *I didn’t think so.* We also have Rovio, a little robot that our remote developers can use to visit the office “virtually.” (There. I said “virtually.” Are you happy now?)Overall 2010 has been a real breakout year for Stack Overflow, which is now the largest programmer website in the world (source: me) and the best, fastest-growing Q&A website in the world (source: also me). We’ve got an incredible team firing on all cylinders, so we’re really looking forward to 2011. <BLOCKQUOTE class=textmessage> Have you been wondering about Distributed Version Control? It has been a huge productivity boon for us, so I wrote Hg Init, a Mercurial tutorial—check it out! </BLOCKQUOTE>You’re reading Joel on Software, stuffed with years and years of completely raving mad articles about software development, managing software teams, designing user interfaces, running successful software companies, and rubber duckies. I’m Joel Spolsky, co-founder of Fog Creek Software, a New York company that proves that you can treat programmers well and still be highly profitable. Programmers get private offices, free lunch, and work 40 hours a week. Customers only pay for software if they’re delighted. We make FogBugz, an enlightened bug tracking and software development tool, Kiln, a distributed source control system that will blow your socks off if you’re stuck on Subversion, and Fog Creek Copilot, which makes remote desktop access easy. I’m also the co-founder of Stack Overflow. <a href="http://www.joelonsoftware.com/items/2011/01/24.html" target="_blank" rel="nofollow">View the original article here</a><div class="blogger-post-footer"></div> </article> <article> <h1>Does your employer own your side projects?</h1> 2011-03-21T16:57:00.001+05:00 There’s a surprising amount of misinformation out there about whether software companies own the work that a programmer does in their spare time.From my answer to the question on answers.onstartups.com:<BLOCKQUOTE dir=ltr>Being an employee of a high tech company whose product is intellectual means that you have decided that you want to sell your intellectual output.</BLOCKQUOTE>Read the whole thing here:<BLOCKQUOTE dir=ltr>If I'm working at a company, do they have intellectual property rights to the stuff I do in my spare time?</BLOCKQUOTE> <BLOCKQUOTE class=textmessage> Have you been wondering about Distributed Version Control? It has been a huge productivity boon for us, so I wrote Hg Init, a Mercurial tutorial—check it out! </BLOCKQUOTE>You’re reading Joel on Software, stuffed with years and years of completely raving mad articles about software development, managing software teams, designing user interfaces, running successful software companies, and rubber duckies. I’m Joel Spolsky, co-founder of Fog Creek Software, a New York company that proves that you can treat programmers well and still be highly profitable. Programmers get private offices, free lunch, and work 40 hours a week. Customers only pay for software if they’re delighted. We make FogBugz, an enlightened bug tracking and software development tool, Kiln, a distributed source control system that will blow your socks off if you’re stuck on Subversion, and Fog Creek Copilot, which makes remote desktop access easy. I’m also the co-founder of Stack Overflow. <a href="http://www.joelonsoftware.com/items/2011/02/11.html" target="_blank" rel="nofollow">View the original article here</a><div class="blogger-post-footer"></div> </article> <article> <h1>Why Payoneer Blocked Pakistan The Truth</h1> 2010-05-04T17:15:00.001+05:00 Link: http://syedaqeel.com/2010/05/why-payoneer-blocked-pakistan-the-truth/ Like every other Payoneer user from Pakistan I became insane after reading support email. However after coming to senses I had done some research about why Payoneer blocked Pakistan. Many readers left comments on earlier post about Payoneer ban for doing protest to Payoneer. <script type="text/javascript"> </script> <script type="text/javascript" src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> </script> Friends you do not need to protest Payoneer, don’t even have to blame them and they can’t help us in this situation. They have to follow policies of US regularity authorities and its what they are doing. Reason for this ban is Pakistani law makers. On March 26th, 2010 Pakistan’s Anti-Money Laundering ordinance of 2008 has expired. So right now Pakistan is a lawless country without any anti-money laundering and anti-terrorist funding laws. Due to that Asia Pacific Group on Financial Action Task Force (FATF) has put Pakistan in category III of risk countries for financial transactions. Due to absence of proper anti-money laundering laws in Pakistan US government issued new regulations for its financial institutions. This is why Payoneer has included Pakistan in banned countries. This is a negligence at end of Pakistani law makers who failed to renew the existing ordinance or approve the new anti-money laundering bill which is already approved by national assembly. According to a story published in TheNews Pakistani senate had a meeting on March 18th in which this law was put forward but the senate “had some other important affairs” to deal with so this law discussion was moved forward to future senate meeting. Chairman Senate Standing Committee on Finance Senator Ahmed Ali said the committee under his chairmanship approved the bill in great haste with conditions that this bill would be tabled again within one year as the committee wanted to bring certain improvements in it. However, he deplored that their hard work could not bring the desired fruit as the Senate was prorogued without passing such an important bill. “Despite our hectic work, the Senate did not pass this bill,” he said and concluded that there was still sufficient time available till March 26 and the government should summon the session of upper house on immediate basis in order to pass this bill in the supreme national interest. No one is our enemy, not Payoneer, not even US government but our own government and law makers. Right now Pakistan officially do not have any anti-money laundering laws. Kudos! Link: http://syedaqeel.com/2010/05/why-payoneer-blocked-pakistan-the-truth/<div class="blogger-post-footer"></div> </article> <article> <h1>Payoneer Blocks its Master Cards for Pakistan</h1> 2010-05-04T16:29:00.001+05:00 By Aamir Attaa · Tuesday, May 4, 2010 5 Comments http://propakistani.pk/2010/05/04/payoneer-blocks-its-master-cards-for-pakistan/ Payoneer, which allows its customers to receive their payments onto a Debit MasterCard that is reloadable through western union, internet (via credit/debit cards) and other similar means, just announced that all Payoneer cards will be blocked for transactions in Pakistan. <script type="text/javascript"> </script> <script type="text/javascript" src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> </script> Payoneer in an email to its customers said that card holders residing in countries listed by the US government as having strategic Anti Money Laundering (AML) deficiencies won’t be able to use Payoneer debit master cards. Other countries on the list include: Angola, Ecuador, Ethiopia, Turkmenistan and Sao Tome and Principe. If your card is in use in one of the countries listed, it is strongly recommended to use all of your funds by May 31, 2010. Payoneer was widely used by freelancers to charge their customer. They used to give a url where customers could pay the bills using their credit cards. This is going to be another blow for countless freelancers of Pakistan, where receiving payments from foreign personals is already very tough. Payoneer sent following email to their customers in Pakistan: Dear Customer, Due to a change in requirements from our regulators, as of June 1, 2010 Payoneer cards will be blocked for transactions in the countries listed by the US government as having strategic Anti Money Laundering (AML) deficiencies. This change will affect card usage in the following countries: Angola, Ecuador, Ethiopia, Pakistan, Turkmenistan and Sao Tome and Principe. If your card is in use in one of the countries listed, we encourage you to use all funds by May 31, while transactions are still possible in these countries. Please note that beyond May 31 your card will continue to be fully functional in all countries not banned by the US government, and we will be happy to load payments to the card without any change. We apologize for the inconvenience and hope that the Payoneer card will continue to provide you with valuable service. Sincerely, Payoneer Customer Support<div class="blogger-post-footer"></div> </article> <article> <h1>Why Payoneer Blocked Pakistan – The Truth</h1> 2010-05-04T16:11:00.002+05:00 Like every other Payoneer user from Pakistan I became insane after reading support email. However after coming to senses I had done some research about why Payoneer blocked Pakistan. Many readers left comments on earlier post about Payoneer ban for doing protest to Payoneer. <script type="text/javascript"> </script> <script type="text/javascript" src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> </script> Friends you do not need to protest Payoneer, don’t even have to blame them and they can’t help us in this situation. They have to follow policies of US regularity authorities and its what they are doing. Reason for this ban is Pakistani law makers. On March 26th, 2010 Pakistan’s Anti-Money Laundering ordinance of 2008 has expired. So right now Pakistan is a lawless country without any anti-money laundering and anti-terrorist funding laws. Due to that Asia Pacific Group on Financial Action Task Force (FATF) has put Pakistan in category III of risk countries for financial transactions. Due to absence of proper anti-money laundering laws in Pakistan US government issued new regulations for its financial institutions. This is why Payoneer has included Pakistan in banned countries. This is a negligence at end of Pakistani law makers who failed to renew the existing ordinance or approve the new anti-money laundering bill which is already approved by national assembly. According to a story published in TheNews Pakistani senate had a meeting on March 18th in which this law was put forward but the senate “had some other important affairs” to deal with so this law discussion was moved forward to future senate meeting. Chairman Senate Standing Committee on Finance Senator Ahmed Ali said the committee under his chairmanship approved the bill in great haste with conditions that this bill would be tabled again within one year as the committee wanted to bring certain improvements in it. However, he deplored that their hard work could not bring the desired fruit as the Senate was prorogued without passing such an important bill. “Despite our hectic work, the Senate did not pass this bill,” he said and concluded that there was still sufficient time available till March 26 and the government should summon the session of upper house on immediate basis in order to pass this bill in the supreme national interest. No one is our enemy, not Payoneer, not even US government but our own government and law makers. Right now Pakistan officially do not have any anti-money laundering laws. Kudos! Link: http://syedaqeel.com/2010/05/why-payoneer-blocked-pakistan-the-truth/<div class="blogger-post-footer"></div> </article> <article> <h1>Payoneer Pakistan Users Can No Longer Use Payoneer – Change in Regulations</h1> 2010-05-04T16:09:00.001+05:00 This is a very bad news for every freelancer and web entrepreneur who was completely relying on Payoneer. In this email one question remains unanswered that users will still be able to receive funds and use Payoneer cards to make transactions to US merchants. I had asked this question to Payoneer support and now waiting for their reply. Will keep my readers updated at this situation. <script type="text/javascript"> </script> <script type="text/javascript" src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> </script> Dear Customer, Due to a change in requirements from our regulators, as of June 1, 2010 Payoneer cards will be blocked for transactions in the countries listed by the US government as having strategic Anti Money Laundering (AML) deficiencies. This change will affect card usage in the following countries: Angola, Ecuador, Ethiopia, Pakistan, Turkmenistan and Sao Tome and Principe. If your card is in use in one of the countries listed, we encourage you to use all funds by May 31, while transactions are still possible in these countries. Please note that beyond May 31 your card will continue to be fully functional in all countries not banned by the US government, and we will be happy to load payments to the card without any change. We apologize for the inconvenience and hope that the Payoneer card will continue to provide you with valuable service. Please feel free to use our Contact Us page for any additional questions. Please do not reply to this e-mail. Sincerely, Payoneer Customer Support<div class="blogger-post-footer"></div> </article> <article> <h1>Upload user specific images in FCK Editor of DNN</h1> 2009-05-20T16:37:00.004+06:00 Question:I would like to be able to let users upload their pictures through the editor "FCKConfig.ImageUpload=true" but i want each user that logs in to be able to upload to their assigned areas for instance, if user1 logs in then when he uploads a picture he would upload it to the user1 in the web server and then when he browses for all pictures using fckeditor he would only be able to see the pictures under the user1 folder. Answer:If you want to be able to set the file browsing and uploading path on the fly for each user or group you will have to use the Application("FCKeditor:UserFilesPath") variable for FCKeditor on the Page_Init event. You can also use the Session("FCKeditor:UserFilesPath"), as a matter of fact, that is what the fckeditor documentation suggests. However, with dotnetnuke i couldnt get it to work properly using the session variable so instead I used the "Application" call, i think that this is the equivalent of using the web.config except that you are doing it dynamically. Anyhow, below is an example of how I am doing it in my site, notice that I am just checking the user name for the current logged in user, I am not checking group memberships like you want: <pre> Protected Sub Page_Init (ByVal sender As Object, ByVal e As System.EventArgs) Handles Me.Init Try Dim userCtrlr As New UserController Dim user As UserInfo = userCtrlr.GetUser(PortalId, UserId) Application("FCKeditor:UserFilesPath") = Page.ResolveUrl("~/Portals/" & PortalId.ToString & "/Userfiles/FlyerImgs/" & user.Username & "/") Catch exc As Exception 'Module failed to load ProcessModuleLoadException(Me, exc) End Try End Sub </pre> Make sure that you are using the "Page.ResolveUrl" otherwise the application will have problems converting the image path, and the image will not display properly. Ref: http://www.dotnetnuke.com/Community/Forums/tabid/795/forumid/127/threadid/158276/scope/posts/Default.aspx<div class="blogger-post-footer"></div> </article> <article> <h1>Urdu textbox JS file</h1> 2009-05-06T16:45:00.002+06:00 Download js file directly. <a href="http://justcsharp.googlepages.com/urdueditor.js">http://justcsharp.googlepages.com/urdueditor.js</a><div class="blogger-post-footer"></div> </article> <article> <h1>Urdu Textbox</h1> 2009-05-06T16:32:00.006+06:00 JS file: <pre><a href="http://justcsharp.googlepages.com/urdueditor.js">http://justcsharp.googlepages.com/urdueditor.js</a></pre> Editor Initialization: Initialize Editor before textbox. <div class="sourceRow">initUrduEditor(""); </div> TextBox: input type="text" onfocus="setEditor(this)" style="font-family: Urdu Naskh Asiatype; font-size: 14px;" id="Editor" name="Editor" lang="ur"<div class="blogger-post-footer"></div> </article> <article> <h1>Restrict Web Page access based on MAC / IP</h1> 2009-04-24T12:17:00.007+06:00 This solution is a combination of .Net / Java applets. Java Applet is used to get the MAC from the client machine and then it is sent back to the server and there it will be used to restrict the client access or grant it access based on the ACL (Access Control List) defined in the XML. Java Applet to Get MAC Address : import java.net.NetworkInterface; import java.net.SocketException; import java.util.Enumeration; import java.util.ArrayList; import java.applet.Applet; public class MacAddressApplet extends Applet { public static String sep = ":"; public static String format = "%02X"; /** * getMacAddress - return the first mac address found * separator - byte seperator default ":" * format - byte formatter default "%02X" * * @param ni - the network interface * @return String - the mac address as a string * @throws SocketException - pass it on */ public static String macToString( NetworkInterface ni ) throws SocketException { return macToString( ni, MacAddressApplet.sep, MacAddressApplet.format ); } /** * getMacAddress - return the first mac address found * * @param ni - the network interface * @param separator - byte seperator default ":" * @param format - byte formatter default "%02X" * @return String - the mac address as a string * @throws SocketException - pass it on */ public static String macToString( NetworkInterface ni, String separator, String format ) throws SocketException { byte mac [] = ni.getHardwareAddress(); if( mac != null ) { StringBuffer macAddress = new StringBuffer( "" ); String sep = ""; for( byte o : mac ) { macAddress.append( sep ).append( String.format( format, o ) ); sep = separator; } /* My extra Code for Vista*/ if (macAddress.length ()! = 0) ( macAddress.toString return (); ) /* End of patch*/ return macAddress.toString(); } return null; } /** * getMacAddress - return the first mac address found * * @return the mac address or undefined */ public static String getMacAddress() { try { Enumeration<networkinterface> nis = NetworkInterface.getNetworkInterfaces(); // not all interface will have a mac address for instance loopback on windows while( nis.hasMoreElements() ) { String mac = macToString( nis.nextElement() ); if( mac != null ) return mac; } } catch( SocketException ex ) { System.err.println( "SocketException:: " + ex.getMessage() ); ex.printStackTrace(); } catch( Exception ex ) { System.err.println( "Exception:: " + ex.getMessage() ); ex.printStackTrace(); } return "undefined"; } /** * getMacAddressesJSON - return all mac addresses found * * @return a JSON array of strings (as a string) */ public static String getMacAddressesJSON() { try { String macs [] = getMacAddresses(); String sep = ""; StringBuffer macArray = new StringBuffer( "['" ); for( String mac: macs ) { macArray.append( sep ).append( mac ); sep = "','"; } macArray.append( "']" ); return macArray.toString(); } catch( Exception ex ) { System.err.println( "Exception:: " + ex.getMessage() ); ex.printStackTrace(); } return "[]"; } /** * getMacAddresses - return all mac addresses found * * @return array of strings (mac addresses) empty if none found */ public static String [] getMacAddresses() { try { Enumeration<networkinterface> nis = NetworkInterface.getNetworkInterfaces(); ArrayList <string> macs = new ArrayList<string>(); while( nis.hasMoreElements() ) { String mac = macToString( nis.nextElement() ); // not all interface will have a mac address for instance loopback on windows if( mac != null ) { macs.add( mac ); } } return macs.toArray( new String[0] ); } catch( SocketException ex ) { System.err.println( "SocketException:: " + ex.getMessage() ); ex.printStackTrace(); } catch( Exception ex ) { System.err.println( "Exception:: " + ex.getMessage() ); ex.printStackTrace(); } return new String[0]; } /** * getMacAddresses - return all mac addresses found * * @param sep - use a different separator */ public static void setSep( String sep ) { try { MacAddressApplet.sep = sep; } catch( Exception ex ) { // don't care } } /** * getMacAddresses - return all mac addresses found * * @param format - the output format string for bytes that can be overridden default hex. */ public static void setFormat( String format ) { try { MacAddressApplet.format = format; } catch( Exception ex ) { // don't care } } public static void main( String... args ) { System.err.println( " MacAddress = " + getMacAddress() ); setSep( "-" ); String macs [] = getMacAddresses(); for( String mac : macs ) System.err.println( " MacAddresses = " + mac ); setSep( ":" ); System.err.println( " MacAddresses JSON = " + getMacAddressesJSON() ); } } Java Applet to Get OS Version : import java.net.*; public class GetOSName extends java.applet.Applet { public String OSName; public String Vendor; public String URL; public String Version; public String OSArch; public String OSVersion; URL location; public void start() { Vendor = System.getProperty("java.vendor"); URL = System.getProperty("java.vendor.url"); Version = System.getProperty("java.version"); OSArch = System.getProperty("os.arch"); OSName = System.getProperty("os.name"); OSVersion = System.getProperty("os.version"); } public String getOSName() { OSName = System.getProperty("os.name"); return OSName; } } The above two applets are used in the CheckAccess.aspx file to get the MAC Address and then the page will autopost back with the MAC Address to the server. CheckAccess.aspx File: In this file add both applets and onload of page call the following functions to get the MAC address and then postback that form back to the server. Add the both applets in the Page.  <embed type="application/x-java-applet" name="macaddressapplet" width="0" height="0" code="MacAddressApplet" archive="macaddressapplet.jar" pluginspage="http://java.sun.com/javase/downloads/index.jsp" style="position:absolute; top:-1000px; left:-1000px;"> <noembed>   <object classid="clsid:CAFEEFAC-0016-0000-FFFF-ABCDEFFEDCBA" type="application/x-java-applet" name="macaddressapplet" style="position:absolute; top:-1000px; left:-1000px;" > <param name="code" value="MacAddressApplet"> <param name="archive" value="macaddressapplet.jar" > <param name="mayscript" value="true"> <param name="scriptable" value="true"> <param name="width" value="0"> <param name="height" value="0"> </object>  </noembed> </embed>   <embed type="application/x-java-applet" name="GetOSName" width="0" height="0" code="GetOSName" archive="GetOSName.jar" pluginspage="http://java.sun.com/javase/downloads/index.jsp" style="position:absolute; top:-1000px; left:-1000px;"> <noembed>   <object classid="clsid:CAFEEFAC-0016-0000-FFFF-ABCDEFFEDCBA" type="application/x-java-applet" name="GetOSName" style="position:absolute; top:-1000px; left:-1000px;" > <param name="code" value="GetOSName"> <param name="archive" value="GetOSName.jar" > <param name="mayscript" value="true"> <param name="scriptable" value="true"> <param name="width" value="0"> <param name="height" value="0"> </object>  </noembed> </embed>  Script to Get MAC Address: <script type="text/javascript"> var macs = { getMacAddress : function() { document.macaddressapplet.setSep( "-" ); //alert( "Mac Address = " + document.macaddressapplet.getMacAddress() ); var myText = document.getElementById("txt_Mac"); myText.value = document.macaddressapplet.getMacAddress() ; }, getMacAddressesJSON : function() { document.macaddressapplet.setSep( "-" ); document.macaddressapplet.setFormat( "%02x" ); var macs = eval( String( document.macaddressapplet.getMacAddressesJSON() ) ); var mac_string = ""; for( var idx = 0; idx < addresses = " + mac_string ); var myText = document.getElementById(" value =" mac_string;" style="font-weight: bold;">Hidden Fields: <input type="text" value="" id="txt_Mac" runat="server" /> <input type="text" value="" id="txt_OS" /> Function to get MAC Adress depending on OS Version: function GetMACAddress() { var OSName = document.GetOSName.getOSName(); var my_OS = document.getElementById("txt_OS"); my_OS.value = OSName; if(OSName == "Windows XP") { macs.getMacAddress(); } else { macs.getMacAddressesJSON(); } } Auto Postback functions: function SubmitForm() { var myText = document.getElementById("txt_Mac"); if(myText.value.length > 0) { var jsVar = "http://localhost:2582/CheckAccess.aspx"; __doPostBack('callPostBack', jsVar); } } CS file: protected void Check_IP_MAC() { XPathDocument xpDoc = new XPathDocument("ACL XML File"); XPathNavigator xpNav = xpDoc.CreateNavigator(); //Gets the IP addresses from the file and compare it with the users IP XPathExpression xpExpression = xpNav.Compile(@"/users/user/ip"); XPathNodeIterator xpIter = xpNav.Select(xpExpression); string _userIP = Request.UserHostAddress.ToString(); string _userMAC = txt_Mac.Value; bool _IPCheck = false; bool _MACCheck = false; while (xpIter.MoveNext()) { if (xpIter.Current.Value == _userIP) { _IPCheck = true; } } //Gets the MAC addresses from the file and compare it with the users MAC xpExpression = xpNav.Compile(@"/users/user/mac"); xpIter = xpNav.Select(xpExpression); while (xpIter.MoveNext()) { if (xpIter.Current.Value == _userMAC) { _MACCheck = true; } } //Sets the Session if the MAC / IP validates if (_IPCheck == true && _MACCheck == true) { Server.Transfer("WEB PAGE LANDING FILE"); } else { Server.Transfer("ERROR FILE"); } } Ref: http://techdetails.blogmatrix.com/:entry:techdetails-2008-02-11-0000/ , http://www.spiration.co.uk/post/1186/Java%20detect%20brower,%20JVM%20vendor,%20Java%20version%20etc</string></string></networkinterface></networkinterface><div class="blogger-post-footer"></div> </article> <article> <h1>Support rich text with the Yahoo! User Interface Library</h1> 2008-12-23T12:42:00.001+05:00 During a recent project, my team's task was to redesign a Web page that utilized an ActiveX control as a rich text editor. One goal of the project was to replace the ActiveX control with a more standardized approach. We chose to use the rich text editor available with the <a href="http://developer.yahoo.com/yui/index.html" target="_blank">Yahoo! User Interface (YUI) Library</a>. This week I examine using the YUI Library's Rich Text Editor. <h2>YUI Library</h2> The <a href="http://articles.techrepublic.com.com/5100-3513-6100361.html" target="_blank">YUI Library</a> is a set of utilities and controls in JavaScript, as well as CSS templates for building richly interactive Web applications using standard technologies such as DHTML, DOM scripting, and AJAX. You can <a href="http://sourceforge.net/project/downloading.php?group_id=165715&filename=yui_2.3.1.zip" target="_blank">download the YUI Library 2.3.1</a> for free from SourceForge.net. The download allows you to install the libraries on a Web server. Another option is to use the library files directly from Yahoo! servers. Yahoo! provides an excellent overview of how to use its servers for applications implementing functionality via the YUI Library. <h2>Rich Text Editor</h2> A recent addition to the YUI Library is the Rich Text Editor. It is a user interface control that replaces the standard HTML textarea element. It allows for the rich formatting of text content, including common structural treatments like lists, formatting treatments like bold and italic text, and drag-and-drop inclusion and sizing of images. A critical feature of the Rich Text Editor is its toolbar, which provides access to various features like text formatting, color choices, and so forth. You may choose which toolbar features to include in an implementation via scripting. In addition, the toolbar is extensible via a plug-in architecture so that advanced implementations can achieve a high degree of customization. <h2>Putting the Rich Text Editor to work</h2> If you want to use the Rich Text Editor, it requires a YUI Library CSS skin to properly render the control. The following YUI Library files (CSS and JavaScript source files) are necessary to use the Rich Text Editor. The following lines reference the files on the previously mentioned Yahoo! servers, but you may use a local installation as well. <pre> <link rel="stylesheet" type="text/css" href="<a href="http://yui.yahooapis.com/2.3.1/build/assets/skins/sam/skin.css">http://yui.yahooapis.com/2.3.1/build/assets/skins/sam/skin.css</a>">  <script type="text/javascript" src="<a href="http://yui.yahooapis.com/2.3.1/build/yahoo-dom-event/yahoo-dom-event.js">http://yui.yahooapis.com/2.3.1/build/yahoo-dom-event/yahoo-dom-event.js</a>"></script> <script type="text/javascript" src="<a href="http://yui.yahooapis.com/2.3.1/build/element/element-beta-min.js">http://yui.yahooapis.com/2.3.1/build/element/element-beta-min.js</a>"></script>  <script src="<a href="http://yui.yahooapis.com/2.3.1/build/container/container_core-min.js">http://yui.yahooapis.com/2.3.1/build/container/container_core-min.js</a>"></script> <script src="<a href="http://yui.yahooapis.com/2.3.1/build/menu/menu-min.js">http://yui.yahooapis.com/2.3.1/build/menu/menu-min.js</a>"></script> <script src="<a href="http://yui.yahooapis.com/2.3.1/build/button/button-beta-min.js">http://yui.yahooapis.com/2.3.1/build/button/button-beta-min.js</a>"></script>  <script src="<a href="http://yui.yahooapis.com/2.3.1/build/editor/editor-beta-min.js">http://yui.yahooapis.com/2.3.1/build/editor/editor-beta-min.js</a>"></script></pre> The following base HTML is used to deliver the Rich Text Box, but you'll still need to add scripting to fully implement the control. <pre><body class="yui-skin-sam"> <textarea name="texteditor" id="texteditor" cols="50" rows="10"> This text will be displayed in the text area. </textarea></body></pre> Now the cols and rows attributes of the HTML texarea element will be overridden by settings specified in the Rich Text Editor's script. The name assigned to the HTML textarea element is important, since it will be used in the JavaScript code when establishing the relationship between a textarea element and a Rich Text Editor. Also, the class assigned to the HTML body element (yui-skin-sam) is used to visually format the Rich Text Editor control. This CSS skin is defined in the CSS file imported into the application (see previous list). The skin should be applied to the parent HTML element of the textarea element. In this case, the HTML body element is used, but it could be any element that contains the textarea. Once the textarea has been defined along with the proper YUI Library files included in the page, the Rich Text Editor control must be rendered. The rendering is accomplished via JavaScript placed within the page. The script can be used to define various Rich Text Editor options such as the toolbar. As an example, I may use the following JavaScript to format our Rich Text Editor: <pre>var rtf = new YAHOO.widget.Editor('texteditor', { height: '300px', width: '522px'}); rtf.render();</pre> This snippet establishes the height and width of the editor while declaring an instance of the Rich Text Editor. Also, it assigns the textarea called texteditor to the Rich Text Editor. The final line in the script actually causes the Rich Text Editor to display when it calls its render method. When you use the Rich Text Editor control without any specifics about the toolbar, it results in the default behavior of including all toolbar features like text alignment, font face, font size, color, and so forth. You may choose to limit the toolbar options available to users. The final example uses a Rich Text Editor, but the toolbar options are defined in the JavaScript code to render it. The following options are used: <ul class="unIndentedList"><li>The toolbar options are defined in its own variable. This is later used to instantiate the editor.</li><li>A title is assigned to the editor via the toolbar's titlebar property.</li><li>The collapse property signals whether the user may collapse/hide the toolbar.</li> <li>The buttons property allows you to define the buttons displayed in the toolbar. In this example, buttons are displayed for text formatting as well as selecting colors.</li></ul> Once the toolbar options are defined, a Rich Text Editor object is instantiated with the toolbar variable passed to it along with the HTML textarea element. The final step is to render the control. A complete list of options is available in the <a href="http://developer.yahoo.com/yui/docs/YAHOO.widget.Editor.html" target="_blank">YUI Library API documentation</a>. <pre><html><head> <title>Working with YUI Library Rich Text Editor</title>  <link rel="stylesheet" type="text/css" href="<a href="http://yui.yahooapis.com/2.3.1/build/assets/skins/sam/skin.css">http://yui.yahooapis.com/2.3.1/build/assets/skins/sam/skin.css</a>" /> <link rel="stylesheet" type="text/css" href="<a href="http://yui.yahooapis.com/2.3.1/build/fonts/fonts-min.css">http://yui.yahooapis.com/2.3.1/build/fonts/fonts-min.css</a>" /> <link rel="stylesheet" type="text/css" href="<a href="http://yui.yahooapis.com/2.3.1/build/container/assets/skins/sam/container.css">http://yui.yahooapis.com/2.3.1/build/container/assets/skins/sam/container.css</a>" /> <link rel="stylesheet" type="text/css" href="<a href="http://yui.yahooapis.com/2.3.1/build/menu/assets/skins/sam/menu.css">http://yui.yahooapis.com/2.3.1/build/menu/assets/skins/sam/menu.css</a>" /> <link rel="stylesheet" type="text/css" href="<a href="http://yui.yahooapis.com/2.3.1/build/button/assets/skins/sam/button.css">http://yui.yahooapis.com/2.3.1/build/button/assets/skins/sam/button.css</a>" /> <link rel="stylesheet" type="text/css" href="<a href="http://yui.yahooapis.com/2.3.1/build/editor/assets/skins/sam/editor.css">http://yui.yahooapis.com/2.3.1/build/editor/assets/skins/sam/editor.css</a>" />  <script type="text/javascript" src="<a href="http://yui.yahooapis.com/2.3.1/build/yahoo-dom-event/yahoo-dom-event.js">http://yui.yahooapis.com/2.3.1/build/yahoo-dom-event/yahoo-dom-event.js</a>"></script> <script type="text/javascript" src="<a href="http://yui.yahooapis.com/2.3.1/build/element/element-beta-min.js">http://yui.yahooapis.com/2.3.1/build/element/element-beta-min.js</a>"></script>  <script src="<a href="http://yui.yahooapis.com/2.3.1/build/container/container_core-min.js">http://yui.yahooapis.com/2.3.1/build/container/container_core-min.js</a>"></script> <script src="<a href="http://yui.yahooapis.com/2.3.1/build/menu/menu-min.js">http://yui.yahooapis.com/2.3.1/build/menu/menu-min.js</a>"></script> <script src="<a href="http://yui.yahooapis.com/2.3.1/build/button/button-beta-min.js">http://yui.yahooapis.com/2.3.1/build/button/button-beta-min.js</a>"></script>  <script src="<a href="http://yui.yahooapis.com/2.3.1/build/editor/editor-beta-min.js">http://yui.yahooapis.com/2.3.1/build/editor/editor-beta-min.js</a>"></script> <style> body { margin:0; padding:0; font-face: arial; font-size: 10pt;} .yui-editor-container {z-index: 999;} .editable { border: 5px solid black; argin-top: 100px; margin: .25em; float: left; width: 350px; height: 100px; overflow: auto; } .textbox { margin-left: 10px; width: 100px; height: 25px; } .label { margin-left: 10px; width: 50px; height: 25px; font-weight: bold; } </style></head> <body class="yui-skin-sam"> First Name:<input type="text" name="firstName" id="firstName" class="textbox" /> Last Name:<input type="text" name="lastName" id="lastName" class="textbox" /> <textarea name="texteditor" id="texteditor"> Test within TextArea. </textarea> <script> var toolbar = { height: '200px', width: '420px', toolbar: { titlebar: 'TechRepublic.com Editor', collapse: true, buttons: [ { group: 'textstyle', label: 'Font Style', buttons: [ { type: 'push', label: 'Bold', value: 'bold' }, { type: 'push', label: 'Italic', value: 'italic' }, { type: 'push', label: 'Underline', value: 'underline' }, { type: 'separator' }, { type: 'select', label: 'Arial', value: 'fontname', disabled: true, menu: [ { text: 'Arial', checked: true }, { text: 'Arial Black' }, { text: 'Comic Sans MS' }, { text: 'Courier New' }, { text: 'Lucida Console' }, { text: 'Tahoma' }, { text: 'Times New Roman' }, { text: 'Trebuchet MS' }, { text: 'Verdana' } ] }, { type: 'spin', label: '13', value: 'fontsize', range: [ 9, 75 ], disabled: true }, { type: 'separator' }, { type: 'color', label: 'Font Color', value: 'forecolor', disabled: true }, { type: 'color', label: 'Background Color', value: 'backcolor', disabled: true } ] } ] } }; myEditor = new YAHOO.widget.Editor('texteditor', toolbar); myEditor.render(); </pre><ul class="postData"><li>Date: November 5th, 2007</li><li>Author: Tony Patton </li> </ul><pre> </script></body></html></pre> It is worth noting that the text is formatted using standard HTML to format the text within the Rich Text Editor, so bold text uses the HTML strong element; the font element is used for font styling, and so forth. <h2>An improved interface</h2> I have been a big fan of the YUI Library since I first discovered it more than a year ago. It allows developers to build powerful Web interfaces using code that has been fully tested for proper functionality. The Rich Text Editor is just one example of the power controls available in the YUI Library. <div class="blogger-post-footer"></div> </article> <article> <h1>Moving to CSS-based layouts with the YUI Library</h1> 2008-12-23T12:41:00.001+05:00 While initiating a recent project to make substantial changes to an existing Web application, it was decided to dump the table-based layout used in its original design in favor of <a target="_blank" href="http://downloads.techrepublic.com.com/download.aspx?docid=307308">CSS</a>. We opted to use the <a target="_blank" href="http://developer.yahoo.com/yui/index.html">Yahoo! User Interface (YUI) Library</a> after evaluating different approaches. The YUI Library provides core CSS resources that have been developed by a professional team of developers and extensively tested by the Web community. This tutorial walks you through the steps of how to move from a table-based design to a CSS-based layout with the help of the YUI Library. <h2>The layout</h2> It is worth considering the division of screen real estate in the application to understand how it is coded using both tables and CSS. The overall page is divided into two horizontal sections: a header and a body. The header portion can be further divided into three horizontal strips. The first strip contains a strip of color at the top. The middle row contains text and a logo on a white background. The final row has its own background color along with a breadcrumb. The body portion of the page is divided into two columns. The first column is a navigation area featuring a list of navigation links. The second column is divided into two rows with a small footer row at the bottom, and the rest is devoted to page content. The site design does not change from the original product developed with Dreamweaver and HTML tables. The introduction of CSS will deliver cleaner code with a smaller footprint and simplified maintenance. <h2>Table-based layout</h2> The original design used six HTML tables to divide the page and deliver the solution. It used CSS to style text on the page, but CSS usage stopped at that point. The following listing includes the HTML source. Background colors are assigned to each table (via the bgcolor attribute) to provide a visual cue of the layout. <pre><html> <head><title>Table-based page layout</title></head> <body> <table bgcolor="yellow" width="100%" height="100%" border="0" cellspacing="0" cellpadding="0"> <tbody> <tr><td colspan="3" width="100%"></td></tr></pre> <pre><tr><td colspan="3"> <table bgcolor="red" border="0" width="100%" cellspacing="0"> <tr> <td width="85%" valign="middle">Header Text</td> <td width="15%"><img src="logo.jpg" mce_src="logo.jpg" /></td> </tr></table></td></tr> <tr> <td width="16%" height="20" nowrap>Breadcrumb</td> <td width="40%" height="20" align="right" nowrap></td> </tr> <tr> <td width="10%" rowspan="2" valign="top"> <table bgcolor="green" width="101%" height="100%" border="0" cellpadding="0" cellspacing="0"> <tr> <td height="167" valign="top"> <table bgcolor="light blue" width="100%" border="0" cellspacing="0"> <tr><td colspan="3">Menu</td></tr> <tr> <td width="8"> </td> <td>Nav Link</td> <td width="4"> </td> </tr></table></td></tr></table></td> <td height="100%" colspan="2" valign="top"> <table bgcolor="brown" width="100%" height="100%" cellpadding="0" cellspacing="0"> <tr><td valign="top"> <table bgcolor="silver" width="96%" height="220" border="0" cellpadding="10"> <tbody><tr> <td valign=top align=left width="100%"> Content goes here </td></tr></tbody></table></td></tr><tr> <td height="40" valign="middle"> Footer </td></tr> </table></td></tr></table></body></html></pre> The table-based layout solution delivers the desired results, but it can be confusing to make layout changes. A quick perusal of the code demonstrates the confusing nature of using tables for layout. It takes time to make layout changes and to convert a table-based solution to a CSS alternative, so selling such a change to a client can be daunting. In our case, the client was technically savvy and easily convinced when we showed the simplified approach offered by CSS. <h2>YUI Grids CSS</h2> The use of the <a target="_blank" href="http://developer.yahoo.com/yui/grids/">YUI Grids CSS</a> feature of the YUI Library added another level of acceptance via a tested solution. YUI Grids CSS provides a CSS solution for delivering page layouts that divide the page into areas. A great aspect of the YUI Grids CSS feature is its <a target="_blank" href="http://developer.yahoo.com/yui/articles/gbs/index.html#history">A-level browser support</a>, which provides the highest support level in terms of browsers. This means you don't have to worry about the quirks in different browsers when using CSS for layout. <h2>CSS layout</h2> YUI Grids CSS offers preset page widths and templates, along with the ability to nest and stack layouts to generate what you need. Yahoo boasts the capability to deliver more than 1,000 layout combinations with it. YUI Grids CSS is part of the <a target="_blank" href="http://developer.yahoo.com/yui/download/">YUI Library download</a>. We used the following features of the YUI Grids CSS feature: <ul><li>The 100% page width is employed via the doc3 id attribute assigned to the overall <DIV> container.</li><li>The entire page is divided into three rows using three <DIV> elements. The YUI Grids CSS standard id attributes for header (hd), body (bd), and footer (ft) are used.</li><li>The header has three rows using two elements and a <DIV> element. The <DIV> includes another <DIV> that uses YUI Grids CSS features. This includes the 100% page width (doc3 attribute), as well as a preset template that has two columns with the narrower column on the left with a width of 180 pixels. The narrower column is assigned the class id of yui-b with the larger column assigned the yui-main attribute. The two columns are used to ensure the breadcrumb appears above the content area of the page.</li><li>The middle or body row of the whole page layout is divided into two columns with a left column width of 180 pixels. This is accomplished with a predefined template employed by assigning the yui-t2 class to the body's <DIV> container. The smaller left column is designated with the yui-b class assignment, and the main area is designated with the yui-main class assignment.</li><li>The footer row uses the same approach as the body with two columns — a left column of 180 pixels.</li><li>The smaller left column of the body row of the page contains a navigation menu. The menu is created with an HTML unordered list and styled via CSS.</li><li>The YUI Grids CSS is contained in one CSS file available in the YUI Library download. The file is called grids.css and has a small footprint of 4KB.</li></ul> Here is the source of the reworked page: <pre><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "<a href="http://www.w3.org/TR/html4/strict.dtd">http://www.w3.org/TR/html4/strict.dtd</a>"></pre> <pre><html> <head> <title>Reworked with YUI Grids CSS</title> <link rel="stylesheet" type="text/css" href="grids.css" mce_href="grids.css"> </head> <body> <div id="doc3"> <div id="hd"> Text <div> <div id="doc3" class="yui-t2"> <div class="yui-b"></div> <div id="yui-main"> <div class="yui-b">Breadcrumb</div> </div> </div> </div> </div> <div id="bd"> <div id="doc3" class="yui-t2"> <div class="yui-b"> <ul class="nav"> <li class="main">Menu</li> <li class="sub">Nav Link</li> </ul> </div> <div id="yui-main"> <div class="yui-b"> Content goes here </div> </div> </div> </div> <div id="ft"> <div id="doc3" class="yui-t2"> <div class="yui-b"> </div> <div id="yui-main"> <div class="yui-b"></pre> <pre>Footer </div></pre> <pre></div></pre> <pre></div></pre> <pre></div></pre> <pre></div></pre> <pre></body></html></pre> You may cringe at the sight of so many <DIV> elements, but this is much easier to follow compared to its table counterpart. Also, the CSS approach allows you to easily modify the layout by editing the CSS or changing what YUI Grids CSS features are used. For example, we could easily modify the layout to use a layout with a left column of 160 pixels by changing the class assigned to the <DIV> elements from yui-t2 to yui-t1. In order to use this approach, you need to be familiar with using YUI Grids CSS. There's a caveat to working with the free CSS elements of the YUI Library: It's tricky to alter the source CSS. The code contains many so-called hacks in order to work with all browsers that you may not be fully aware of all of them when you're editing the CSS. For me, I avoid working directly with the source and work within the confines of the YUI Library. <h2>Making the switch</h2> CSS has matured to the point where using it for layout is now acceptable. However, this approach does have pitfalls, which include browser quirks. For this reason, I find the freely available YUI Grids CSS portion of the YUI Library to be an excellent resource for building Web interfaces that use Web standards. Are you embracing CSS for Web page layout? Do you use any aspect of the YUI Library in your applications? Share your experiences with the Web Developer community. <h2>Additional TechRepublic resources about the YUI Library</h2> <ul><li><a target="_blank" href="http://articles.techrepublic.com.com/5100-3513-6100361.html">Build powerful Web applications via the Yahoo! Developer Network</a></li><li><a target="_blank" href="http://articles.techrepublic.com.com/5100-10878_11-6102360.html">Get reusable CSS at no cost from Yahoo!</a></li> <li><a target="_blank" href="http://blogs.techrepublic.com.com/programming-and-development/?p=541http://blogs.techrepublic.com.com/programming-and-development/?p=541">Support rich text with the Yahoo! User Interface Library</a></li> </ul> <ul class="postData"><li>Date: March 17th, 2008</li><li>Author: Tony Patton </li></ul> <div class="blogger-post-footer"></div> </article> <article> <h1>Build directory structures using SQL Server 2005</h1> 2008-12-23T12:40:00.001+05:00 If you ever work with directory structures on the filing system, you know how challenging it can be to traverse through folders to find specific file(s). If you store this type of information structure in the database, you are even more aware of what it takes to retrieve the data. Writing queries to pull this information is sometimes difficult to achieve and inefficient. You can use the <a target="_blank" href="http://blogs.techrepublic.com.com/datacenter/?p=275">recursion</a> and <a target="_blank" href="http://articles.techrepublic.com.com/5100-9592-6140404.html">XML features</a> in SQL Server 2005 to build a file location on the fly. <h2>Example</h2> This example searches for a document and builds the path to the document based upon a parent-folder to child-folder relationship in the database. One file will belong to one folder, which may be a child folder in a long lineage of parent folders. The ultimate goal is to provide a file to be searched for, and the process will build the location to the file. I've seen file path locations stored several ways in the database, usually with the purpose of storing the location of a file to be pulled for a Web site. Most of the time, the full path to a file is stored in one database field, but I have also seen the location of a file "<a target="_blank" href="http://articles.techrepublic.com.com/5100-9592_11-6135910.html">normalized</a>" so that the past must be built when needed. My goal for this article is to solve the issue of building the path from the hierarchical structure. The script below creates a Documents table and a Folders table. The Documents table stores the filenames and the folder that the document resides in. The Folders table stores the directory structure of one or more local or network drives. Most of the work in this example will involve traversing through this folder structure to build the path to the file. <pre>IF OBJECT_ID('Documents','U') IS NOT NULL DROP TABLE Documents IF OBJECT_ID('Folders','U') IS NOT NULL DROP TABLE Folders IF OBJECT_ID('udf_BuildDocumentPath','FN') IS NOT NULL DROP FUNCTION udf_BuildDocumentPath CREATE TABLE Documents ( DocumentID SMALLINT, FolderID SMALLINT, DocumentName VARCHAR(255) ) CREATE TABLE Folders ( FolderID SMALLINT, ParentFolderID SMALLINT, FolderName VARCHAR(255) )</pre> The code below adds data to our newly created tables. I am adding data for three documents, all of which are located in the same folder. <pre>INSERT INTO Documents(DocumentID, FolderID, DocumentName) VALUES(1,5,'SalesForecast2008.xls') INSERT INTO Documents(DocumentID, FolderID, DocumentName) VALUES(2,5,'SalesProjection.doc') INSERT INTO Documents(DocumentID, FolderID, DocumentName) VALUES(3,5,'SalesForecastPresentation.ppt') INSERT INTO Folders(FolderID, ParentFolderID, FolderName) VALUES(1,null, 'D:') INSERT INTO Folders(FolderID, ParentFolderID, FolderName) VALUES(2,1, 'Sales') INSERT INTO Folders(FolderID, ParentFolderID, FolderName) VALUES(3,2, 'Forecasts') INSERT INTO Folders(FolderID, ParentFolderID, FolderName) VALUES(4,3, 'Data') INSERT INTO Folders(FolderID, ParentFolderID, FolderName) VALUES(5,4, '2008') GO</pre> The script below creates the function that will build the full path to the file based upon the DocumentID in the Documents table. This function uses a recursive common table expression (CTE) to traverse through the directory structure, linking the child folder ID to the parent folder ID in the table. Once the set of records are found that comprise the full path to the document, the FOR XML PATH('') construct is used to "pivot" these values from values in different rows to values concatenated in the same row. From there, it is just a matter of returning the build path to the caller. FOR XML Path() is one of my favorite features in SQL Server 2005 because it makes it so easy to take a list of column values from different rows and concatenate them together so that they are on the same row. It is an ideal tool for <a target="_blank" href="http://blogs.techrepublic.com.com/datacenter/?p=306">dynamically building SQL statements</a> that require list of different values for use in an IN() statement. <pre>CREATE FUNCTION udf_BuildDocumentPath ( @DocumentID SMALLINT ) RETURNS VARCHAR(400) AS BEGIN DECLARE @ReturnPath VARCHAR(400) ;WITH DirectoryPathCTE(DocumentID, FolderID, ParentFolderID, DocumentName, FolderName, LevelNumber) AS ( SELECT DocumentID, f.FolderID, ParentFolderID, DocumentName, f.FolderName, 0 FROM Documents d INNER JOIN folders f on d.FolderID = f.FolderID WHERE DocumentID = @DocumentID UNION ALL SELECT DocumentID, f.FolderID, f.ParentFolderID, DocumentName, f.FolderName, p.LevelNumber + 1 FROM Folders f INNER JOIN DirectoryPathCTE p on p.ParentFolderID = f.FolderID ) SELECT @ReturnPath = ( SELECT FolderName + '' + CASE WHEN LevelNumber = 0 THEN DocumentName ELSE '' END FROM DirectoryPathCTE p ORDER BY LevelNumber DESC FOR XML PATH('') ) RETURN(@ReturnPath) END GO</pre> Now that my function is built, I can call it for every document I have in my Documents table, and the path to the file will be built based upon the DocumentID in the Documents table. <pre>SELECT dbo.udf_BuildDocumentPath(d.DocumentID) FROM Documents d</pre> <h2>Conclusion</h2> Even if you never need to implement the example in this article, I hope you'll take away some ideas from the recursion and the FOR XML PATH clause to solve some tricky problems you may encounter in the future. <ul class="postData"><li>Date: March 17th, 2008</li><li>Author: Tim Chapman </li></ul> <div class="blogger-post-footer"></div> </article> <article> <h1>Adapt to your audience with CSS media types</h1> 2008-12-23T12:37:00.001+05:00 In addition to worrying about browsers, Web standards, and more, you also have to consider the multitude of devices or media that may be used to view a site. The CSS media type provides the functionality to build Web applications whose presentation may vary with its target media. Here's an examination of this CSS feature, along with examples. <h2>The media</h2> A CSS media type names a set of CSS properties. A user agent that claims to support a media type by name must implement all of the properties that apply to that media type. Style sheets allow you to specify how a document is presented with different media. For example, viewing a page on the screen should be presented differently than when it is printed or spoken via a speech synthesizer for the visually impaired. Here's a list of the media types that are currently supported: <ul><li>all: Applies to all media types.</li><li>aural: Used for speech and sound synthesizers.</li><li>braille: Provides support for braille tactile feedback devices.</li> <li>embossed: Used for paged braille printers.</li><li>handheld: Used to target small or handheld devices with limited screen space.</li><li>print: Applies to printers, so users can easily print a page's content.</li> <li>projection: Used for projected presentations like slides.</li><li>screen: The most common delivery mechanism for Web content: the computer screen.</li><li>tty: Used for media using a fixed-pitch character grid, like teletypes and terminals.</li> <li>tv: This applies to television-like devices such as <a target="_blank" href="http://www.webtv.com/pc/">MSN TV</a>.</li></ul> With these media types in mind, you can develop separate style sheets to use for the different media that may be used by users who visit your site. <h2>Media-specific CSS</h2> You may specify different media types a variety of ways. First, you may use @import at-rules that specify the media type for an external style sheet that is imported, as the following line accomplishes with importing a special style sheet for tv-based devices. <pre>@import url("webtv.css") tv;</pre> You may use the @media rule to provide presentation rules for handling certain media types within a Web page. The following lines show how this could be used to use a white background for tv-like devices: <pre>@media tv { background: #fff; }</pre> The Link HTML element allows you to link to external style sheets and use the media attribute to specify the target media type for the link. The following snippet shows how it may be used: <pre><LINK rel="stylesheet" type="text/css" media="tv" href="tv.css" mce_href="tv.css"></pre> A caveat of each approach is that you may specify more than one media type for a rule by separating the individual media types with commas. For example, the previous HTML snippet can be rewritten to target tv-like media as well as projection media with the following line. <pre><LINK rel="stylesheet" type="text/css" media="tv, projection" href="tv.css" mce_href="tv.css"></pre> <h2>In action</h2> Different media types are supported so you can format content differently for the various device or media types available. A good example is printing where items like menus and sidebars are omitted (most often using display: none) in the printed output. (Check out Shawn Morton's article about <a target="_blank" href="http://articles.techrepublic.com.com/5100-6329_11-5035184.html">building printer-friendly pages</a>.) The following CSS is included as a brief example of formatting content for different media types. The background is blue for regular screens and white for printed or handheld output. Also, the font size is set to easier to read 12 point for printed output and smaller for handhelds. The font family is set to apply to all media types. <pre><style type="text/css"> @nedia all { font-family: sans-serif; } @media handheld { body { border: none; padding-bottom: 5px; font-size: 8pt; background: white; } h1, h2 { font-size: 10pt;; } } @media screen { body { background: blue; font-size: 10pt; } } @media print { body { width: auto; margin: 0 5%; padding: 0; font-size: 12pt; } } </style></pre> The size of the style sheet for each media type will be much larger than this simple example in real-world applications, so placing each media type's code in separate files and using the Link element will save bandwidth with less code to download. A key issue with any Web standard is its support within the community. <h2>Support</h2> CSS media types are a part of the <a target="_blank" href="http://www.w3.org/TR/REC-CSS2/">CSS2</a> standard, but browser adoption and compliance has been very slow. The media type values of all, screen, and print are well supported, but the remaining types have only received limited support within the browser market. You should test vigorously to ensure the results match expectations with target platforms. At this time, the handheld media type is supported by some devices and software, so you should test with target devices to measure support. The Opera browser supports the projection media type, but support outside of Opera is limited. Support for the other media types like braille, tv, tty, embossed, and aural is almost non-existent. The <a target="_blank" href="http://www.cs.cornell.edu/Info/People/raman/emacspeak/emacspeak.html">Emacspeak</a> browser utilizes the aural media type. <h2>Know your audience</h2> The CSS2 specification includes the media type feature, which allows developers to tailor content for certain media types via style sheets. The feature has been adopted in a piecemeal fashion by the Web community as the print and screen types are fully supported, while support for others is inconsistent and sometimes non-existent. Meanwhile, the <a target="_blank" href="http://www.w3.org/TR/css3-roadmap/">CSS3</a> standard moves forward with the goal of <a target="_blank" href="http://www.w3.org/TR/css3-mediaqueries/">enhancing the media type feature</a>. Have you included media type support in your Web applications? Share your thoughts and experiences with CSS and media types with the Web developer community by posting to the discussion. <h2>Additional CSS resources from TechRepublic</h2> <ul><li><a target="_blank" href="http://blogs.techrepublic.com.com/programming-and-development/?p=401">CSS 101: Styling the cursor</a></li><li><a target="_blank" href="http://blogs.techrepublic.com.com/programming-and-development/?p=419">Control element placement with CSS stacking</a></li> <li><a target="_blank" href="http://blogs.techrepublic.com.com/howdoi/?p=151">How do I… Control the format of HTML lists with CSS?</a></li><li><a target="_blank" href="http://downloads.techrepublic.com.com/download.aspx?docid=307308">Download: CSS Reference Table</a></li> </ul> <ul class="postData"><li>Date: April 4th, 2008</li><li>Author: Tony Patton </li></ul> <div class="blogger-post-footer"></div> </article> <article> <h1>How do I… Create a scrolling content box in CSS?</h1> 2008-12-23T12:36:00.001+05:00 In my last <a href="http://blogs.techrepublic.com.com/programming-and-development/?p=633" target="_blank">TechRepublic Programming and Development blog</a> entry, I showed you how to use Flash to create an animation that mimicked the behavior of the retired (and sometimes reviled) <marquee> tag. I had mentioned that it was possible to use <a href="http://search.techrepublic.com.com/index.php?t=1&s=0&o=0&q=css" target="_blank">CSS</a> to duplicate the <marquee> tag as well, but only if it were coupled with a few lines of knotty JavaScript. However, there is an alternative CSS markup that will let you confine large blocks of content to a set width and height and let you scroll through them using a scrollbar control. This gives the effect of viewing content in a frame (remember those?), but the scrolling box stays put on one page and you can place it anywhere in your layout that you wish. This gives both you and your clients much more control over how you view your Web pages. All of my clients like this layout solution, and I bet yours will as well. This blog post is also available in PDF format as a <a href="http://downloads.techrepublic.com.com/abstract.aspx?docid=351960" target="_blank">TechRepublic download</a>, which includes a sample Web page and the example code. <h2>Start</h2> Open your preferred HTML editor and create a new page. Enter the code in Listing A to get started. <h3>Listing A</h3> <pre><html> <head> <title>Scrolling Content Box</title> <body> </body> </html></pre> Let's go ahead and place some formatted text in the Body of the HTML document, so we can see the effects of our CSS code as we build it (Listing B). Place a significant amount of text into the page. This will give us plenty of text to scroll through when the page is complete. <h3>Listing B</h3> <pre><html> <head> <title> Scrolling Content Box </title> </head> <body> "It had a perfectly round door like a porthole, painted green, with a shiny yellow brass knob in the exact middle. The door opened on to a tube-shaped hall like a tunnel: a very comfortable tunnel without smoke, with paneled walls and floors tiled and carpeted, provided with polished chairs, and lots and lots of pegs for hats and coats--the hobbit was fond of visitors. The tunnel wound on and on, going fairly but not quite straight into the side of the hill--The Hill, as all the people for many miles round called it--and many little round doors opened out of it, first on one side and then on another. No going upstairs for the hobbit: bedrooms, bathrooms, cellars, pantries (lots of these), wardrobes (he had whole rooms devoted to clothes), kitchens, dining-rooms, all were on the same floor, and indeed on the same passage. The best rooms were all on the lefthand side (going in), for these were the only ones to have windows, deep-set round windows looking over his garden and meadows beyond, sloping down to the river." </body> </html></pre> Our text is in place, but let's go ahead and assign the CSS class to it. Even though the CSS markup has not been coded yet, by assigning the style to the text, you can preview the HTML you are writing at any time to see how the styling is built step by step. We'll name our style class "scrollBox" in a couple of steps, so assign a <div> tag with that attribute as shown in Listing C. <h3>Listing C</h3> <pre><html> <head> <title> Scrolling Content Box </title> </head> <body> <div class="scrollBox"> "It had a perfectly round door like a porthole, painted green, with a shiny yellow brass knob in the exact middle. The door opened on to a tube-shaped hall like a tunnel: a very comfortable tunnel without smoke, with paneled walls and floors tiled and carpeted, provided with polished chairs, and lots and lots of pegs for hats and coats--the hobbit was fond of visitors. The tunnel wound on and on, going fairly but not quite straight into the side of the hill--The Hill, as all the people for many miles round called it--and many little round doors opened out of it, first on one side and then on another. No going upstairs for the hobbit: bedrooms, bathrooms, cellars, pantries (lots of these), wardrobes (he had whole rooms devoted to clothes), kitchens, dining-rooms, all were on the same floor, and indeed on the same passage. The best rooms were all on the lefthand side (going in), for these were the only ones to have windows, deep-set round windows looking over his garden and meadows beyond, sloping down to the river." </div> </body> </html></pre> Now that our content is in place, it's time to create your CSS code in the head of the document Declare the style and place it immediately after the closing </TITLE> tag. (Listing D) <h3>Listing D</h3> <pre><html> <head> <title> Scrolling Content Box </title> <style type="text/css">  </style> </head> <body> <div class="scrollBox"> "It had a perfectly round door like a porthole, painted green, with a shiny yellow brass knob in the exact middle. The door opened on to a tube-shaped hall like a tunnel: a very comfortable tunnel without smoke, with paneled walls and floors tiled and carpeted, provided with polished chairs, and lots and lots of pegs for hats and coats--the hobbit was fond of visitors. The tunnel wound on and on, going fairly but not quite straight into the side of the hill--The Hill, as all the people for many miles round called it--and many little round doors opened out of it, first on one side and then on another. No going upstairs for the hobbit: bedrooms, bathrooms, cellars, pantries (lots of these), wardrobes (he had whole rooms devoted to clothes), kitchens, dining-rooms, all were on the same floor, and indeed on the same passage. The best rooms were all on the lefthand side (going in), for these were the only ones to have windows, deep-set round windows looking over his garden and meadows beyond, sloping down to the river." </div> </body> </html></pre> The next step is to declare a class name for the scrolling box style. Let's keep things simple and name it "scrollBox" by typing in the new code in Listing E. <h3>Listing E</h3> <pre><html> <head> <title> Scrolling Content Box </title> <style type="text/css"> <!- .scrollBox { } --> </style> </head> <body> <div class="scrollBox"> "It had a perfectly round door like a porthole, painted green, with a shiny yellow brass knob in the exact middle. The door opened on to a tube-shaped hall like a tunnel: a very comfortable tunnel without smoke, with paneled walls and floors tiled and carpeted, provided with polished chairs, and lots and lots of pegs for hats and coats--the hobbit was fond of visitors. The tunnel wound on and on, going fairly but not quite straight into the side of the hill--The Hill, as all the people for many miles round called it--and many little round doors opened out of it, first on one side and then on another. No going upstairs for the hobbit: bedrooms, bathrooms, cellars, pantries (lots of these), wardrobes (he had whole rooms devoted to clothes), kitchens, dining-rooms, all were on the same floor, and indeed on the same passage. The best rooms were all on the lefthand side (going in), for these were the only ones to have windows, deep-set round windows looking over his garden and meadows beyond, sloping down to the river." </div> </body> </html></pre> Now we have to assign some attributes to the scrollBox class. Let's start by defining how the text itself should look. Helvetica is back in style, and 10 pixels high with a line height of 12 pixels should look fine. <h3>Listing F</h3> <pre><html> <head> <title> Scrolling Content Box </title> <style type="text/css"> <!- .scrollBox { font-family: Helvetica, sans-serif; font-size: 10px; line-height: 12px; } --> </style> </head> <body> <div class="scrollBox"> "It had a perfectly round door like a porthole, painted green, with a shiny yellow brass knob in the exact middle. The door opened on to a tube-shaped hall like a tunnel: a very comfortable tunnel without smoke, with paneled walls and floors tiled and carpeted, provided with polished chairs, and lots and lots of pegs for hats and coats--the hobbit was fond of visitors. The tunnel wound on and on, going fairly but not quite straight into the side of the hill--The Hill, as all the people for many miles round called it--and many little round doors opened out of it, first on one side and then on another. No going upstairs for the hobbit: bedrooms, bathrooms, cellars, pantries (lots of these), wardrobes (he had whole rooms devoted to clothes), kitchens, dining-rooms, all were on the same floor, and indeed on the same passage. The best rooms were all on the lefthand side (going in), for these were the only ones to have windows, deep-set round windows looking over his garden and meadows beyond, sloping down to the river." </div> </body> </html></pre> The font is typeset to satisfaction. The next step is to determine how wide and tall the containing box will be. Type in the code in Listing G to give the box a width of 200 pixels and a height of 150 pixels. <h3>Listing G</h3> <pre><html> <head> <title> Scrolling Content Box </title> <style type="text/css"> <!- .scrollBox { font-family: Helvetica, sans-serif; font-size: 10px; line-height: 12px; height: 150px; width: 200px; } --> </style> </head> <body> <div class="scrollBox"> "It had a perfectly round door like a porthole, painted green, with a shiny yellow brass knob in the exact middle. The door opened on to a tube-shaped hall like a tunnel: a very comfortable tunnel without smoke, with paneled walls and floors tiled and carpeted, provided with polished chairs, and lots and lots of pegs for hats and coats--the hobbit was fond of visitors. The tunnel wound on and on, going fairly but not quite straight into the side of the hill--The Hill, as all the people for many miles round called it--and many little round doors opened out of it, first on one side and then on another. No going upstairs for the hobbit: bedrooms, bathrooms, cellars, pantries (lots of these), wardrobes (he had whole rooms devoted to clothes), kitchens, dining-rooms, all were on the same floor, and indeed on the same passage. The best rooms were all on the lefthand side (going in), for these were the only ones to have windows, deep-set round windows looking over his garden and meadows beyond, sloping down to the river." </div> </body> </html></pre> If we were to just leave the styling alone here, the type would butt up against the scrollbar when it is viewed in the Web browser. Enter the additional code in Listing H to give the type some padding along all edges of the containing box. <h3>Listing H</h3> <pre><html> <head> <title> Scrolling Content Box </title> <style type="text/css"> <!- .scrollBox { font-family: Helvetica, sans-serif; font-size: 10px; line-height: 12px; height: 150px; width: 200px; padding: 5px; } --> </style> </head> <body> <div class="scrollBox"> "It had a perfectly round door like a porthole, painted green, with a shiny yellow brass knob in the exact middle. The door opened on to a tube-shaped hall like a tunnel: a very comfortable tunnel without smoke, with paneled walls and floors tiled and carpeted, provided with polished chairs, and lots and lots of pegs for hats and coats--the hobbit was fond of visitors. The tunnel wound on and on, going fairly but not quite straight into the side of the hill--The Hill, as all the people for many miles round called it--and many little round doors opened out of it, first on one side and then on another. No going upstairs for the hobbit: bedrooms, bathrooms, cellars, pantries (lots of these), wardrobes (he had whole rooms devoted to clothes), kitchens, dining-rooms, all were on the same floor, and indeed on the same passage. The best rooms were all on the lefthand side (going in), for these were the only ones to have windows, deep-set round windows looking over his garden and meadows beyond, sloping down to the river." </div> </body> </html></pre> One last bit of graphic design before we move on to the scroll functionality. Give the box a light blue background using the additional markup shown in Listing I. <h3>Listing I</h3> <pre><html> <head> <title> Scrolling Content Box </title> <style type="text/css"> <!- .scrollBox { font-family: Helvetica, sans-serif; font-size: 10px; line-height: 12px; height: 150px; width: 200px; padding: 5px; background-color: #CCCCFF; } --> </style> </head> <body> <div class="scrollBox"> "It had a perfectly round door like a porthole, painted green, with a shiny yellow brass knob in the exact middle. The door opened on to a tube-shaped hall like a tunnel: a very comfortable tunnel without smoke, with paneled walls and floors tiled and carpeted, provided with polished chairs, and lots and lots of pegs for hats and coats--the hobbit was fond of visitors. The tunnel wound on and on, going fairly but not quite straight into the side of the hill--The Hill, as all the people for many miles round called it--and many little round doors opened out of it, first on one side and then on another. No going upstairs for the hobbit: bedrooms, bathrooms, cellars, pantries (lots of these), wardrobes (he had whole rooms devoted to clothes), kitchens, dining-rooms, all were on the same floor, and indeed on the same passage. The best rooms were all on the lefthand side (going in), for these were the only ones to have windows, deep-set round windows looking over his garden and meadows beyond, sloping down to the river." </div> </body> </html></pre> Now, if you previewed your code thus far in a Web browser, you would see that the enormous block of text would be in a blue box, but it would be cut off after several lines. This is due to the height attribute. Therefore, we need a line of markup (shown in Listing J) that tells the browser to allow us to see the text beyond that height limit. This is done with the overflow rule and we'll set its attribute to auto, which will tell the browser to show the scrollbar controls as needed. <h3>Listing J</h3> <pre><html> <head> <title> Scrolling Content Box </title> <style type="text/css"> <!- .scrollBox { font-family: Helvetica, sans-serif; font-size: 10px; line-height: 12px; height: 150px; width: 200px; padding: 5px; background-color: #CCCCFF; overflow: auto; } --> </style> </head> <body> <div class="scrollBox"> "It had a perfectly round door like a porthole, painted green, with a shiny yellow brass knob in the exact middle. The door opened on to a tube-shaped hall like a tunnel: a very comfortable tunnel without smoke, with paneled walls and floors tiled and carpeted, provided with polished chairs, and lots and lots of pegs for hats and coats--the hobbit was fond of visitors. The tunnel wound on and on, going fairly but not quite straight into the side of the hill--The Hill, as all the people for many miles round called it--and many little round doors opened out of it, first on one side and then on another. No going upstairs for the hobbit: bedrooms, bathrooms, cellars, pantries (lots of these), wardrobes (he had whole rooms devoted to clothes), kitchens, dining-rooms, all were on the same floor, and indeed on the same passage. The best rooms were all on the lefthand side (going in), for these were the only ones to have windows, deep-set round windows looking over his garden and meadows beyond, sloping down to the river." </div> </body> </pre><ul class="postData"><li>Date: April 10th, 2008</li><li>Author: John Lee </li></ul><pre> </html></pre> Preview the page in several browsers across operating systems to see how they display the design markup and scrollbar controls. Since scrollBox is a CSS class, it can be styled even further and it can be applied to any HTML element, not just blocks of text. Try applying it to <img> tags, for example, to create scroll box containers for high-resolution images. <div class="blogger-post-footer"></div> </article> <article> <h1>Create custom Visual Studio 2005 code snippets</h1> 2008-12-23T12:35:00.001+05:00 <a target="_blank" href="http://blogs.techrepublic.com.com/programming-and-development/?p=655">Visual Studio 2005's built-in code snippets</a> are great, but you can extend the concept by creating custom code snippets to easily reuse common code blocks in your application. This tutorial walks you through the process of creating custom code snippets. <h2>Creating the file</h2> Code snippets are defined in XML files with the .snippet file extension. Microsoft created the <a target="_blank" href="http://msdn2.microsoft.com/en-us/library/ms171418.aspx">Code Snippet XML Schema</a>, which defines the structure of a snippet XML file and specifies the elements (and their attributes) that you may use in a code snippet file. Since code snippets are defined as XML, you can use your favorite XML or text editor to create and maintain them. Visual Studio 2005 supports XML editing, so it may be used to remain in a common environment. <h2>Code snippet example</h2> Depending on the project or organization, many code snippet candidates may exist. This may be a discrete chunk of code that you may reuse throughout the code (this also lends itself to a code library). In addition, the code may or may not include replaceable parameters that allow you to enforce a coding technique. To demonstrate, I will create a code snippet that declares a set of objects for working with SQL Server. This allows a developer to insert and use these objects while saving typing time. Here is the C# code snippet: <pre>SqlConnection conn;</pre> <pre>SqlCommand comm;</pre> <pre>SqlDataAdapter sda;</pre> <a target="_blank" href="http://techrepublic.com.com/html/tr/sidebars/6067392-0.html">Listing A</a> contains the appropriate code snippet XML. <a target="_blank" href="http://techrepublic.com.com/html/tr/sidebars/6067392-1.html">Listing B</a> contains the equivalent code snippet definition for <a href="http://VB.NET">VB.NET</a>. Here's a closer look at the XML: <ul><li>A code snippet is defined by the CodeSnippet element. The CodeSnippets element is the root node, and it may contain one or more CodeSnippet elements.</li><li>The Description, Author, and Shortcut elements appear in the Code Snippets Manager window when it is highlighted. For this reason, the text included in the Description element should give a good idea of what the snippet actually does. In addition, the Shortcut element is what is typed within Visual Studio 2005 to actually use the snippet (type the shortcut and hit [Tab]).</li><li>The Code element contains the actual code associated with the snippet. In this example, there are no replaceable parameters used. The code is placed within the CDATA attribute.</li><li>The Language attribute of the Code element defines the code used. The options include VB, CSharp, VJSharp, and XML.</li></ul> You should create a folder to contain your custom code snippets and save this sample in the folder. The next step is to make it available in Visual Studio 2005. <h2>Adding a snippet to Visual Studio 2005</h2> The Code Snippets Manager allows you to easily add your own code snippets to the environment. To do so, you should follow these steps: <ol><li>Open the Manager window via the Tools | Code Snippets Manager menu selection.</li><li>Select the appropriate language in the Language field.</li><li>Click the Add button to add your snippet to the environment. This opens the Code Snippets Directory window that allows you to navigate to the folder containing the snippet file(s). You don't select the actual snippet file — the folder is selected and all snippets within it are added.</li><li>Once the directory is added, you can select it and browse the files within it.</li></ol> You can test it by typing the appropriate shortcut in Visual Studio 2005 and hitting [Tab] to insert the code, or you can right-click and select Insert Snippet from the context menu. This allows you to navigate the available code snippet directories to locate one if you don't know its shortcut. If you're using C#, you may also use the Surround With context menu option. This is just one way to tackle this simple example. <h2>More examples</h2> The Code Snippet XML Schema includes the optional Declarations element that you may use to specify the literals and objects for the code specified in the snippet. It might include zero or more Literal elements that you may use to define literals, and zero or more Object elements to define objects used in the code. Literal and Object elements contain ID, ToolTip, and Default elements to assign a name to the item, tooltip text, and its default value. In addition, the Object element contains the Type element to specify the type of object. The Object and Literal elements are used in the actual code within the Code element. You reference the literals and objects you declared in the Declarations element by placing $ symbols at the beginning and end of the value in the literal or object's ID element. The VB snippet file in <a target="_blank" href="http://techrepublic.com.com/html/tr/sidebars/6067392-2.html">Listing C</a> uses these elements to declare literals to be used in our code. When you use this code snippet, green boxes appear around the literals enclosed in dollar signs ($connString$ and $sqlString$), which allow you to insert the appropriate text. The example code snippets in this column provide a peek at the creation process, but there is much more functionality available via the Code Snippet XML Schema. <h2>Importing code snippets</h2> While creating custom code snippets provides power and flexibility, you may also import snippets (created by you or others) to Visual Studio 2005 via the Import button in the Code Snippets Manager. This allows you to bring snippets into existing directories used by the Code Snippets Manager. You select the destination directories and select the Finish button. The process physically copies the snippet file to the destination directory. Microsoft provides plenty of <a target="_blank" href="http://msdn2.microsoft.com/en-us/vs2005/aa718338.aspx">Visual Studio 2005 code snippets</a>, which you can download and import into your environment. <h2>Third-party tools</h2> It can be a bit tedious to create and edit XML files. It is especially cumbersome when you must know the syntax and order of a valid element that you may use in the snippet file. While Visual Studio 2005 does not include a snippet editor, there are code snippet editor tools available: <ul><li><a target="_blank" href="http://www.codeplex.com/snippy">Snippy — Visual Studio Code Snippet Editor</a></li><li><a target="_blank" href="http://blogs.msdn.com/gusperez/articles/93681.aspx">Snippy — C# Code Snippet Editor</a></li> </ul> <h2>Customize your environment</h2> Creating custom code snippets allows you to customize and streamline your development environment by making it easy to reuse standard coding elements and save typing. Use your favorite XML or text editor to create your own snippets and add them to Visual Studio 2005 to save valuable time. <ul class="postData"><li>Date: April 18th, 2008</li><li>Author: Tony Patton </li></ul> <div class="blogger-post-footer"></div> </article> <article> <h1>Visual Studio 2008 simplifies JavaScript debugging</h1> 2008-12-23T12:34:00.001+05:00 One of the more cumbersome development tasks is debugging client-side JavaScript code. Tools such as <a target="_blank" href="http://blogs.techrepublic.com.com/programming-and-development/?p=477">Firebug</a> are helpful, but in the case of Firebug, you're forced to use Firefox. Thankfully, Visual Studio 2008 provides a robust and developer-friendly environment for debugging JavaScript. <h2>Simplification</h2> Whenever I hear developers discuss the JavaScript debugger features in Visual Studio 2008, I notice that they're quick to point out that this type of functionality has been available in prior versions of the Visual Studio IDE for a long time. This is certainly true, but the improvements in Visual Studio 2008 make it much simpler to use. The irritating aspect of <a target="_blank" href="http://articles.techrepublic.com.com/5100-10878_11-6067987.html">JavaScript debugging</a> in Visual Studio 2005 is that you have to first execute the application before any breakpoints can be created in your JavaScript. This approach has been modified in Visual Studio 2008 to allow breakpoints to be defined in the source code before the application runs. In fact, you can set them in both your C#/<a href="http://VB.NET">VB.NET</a> code as well as JavaScript. So, when the application is run, you can step through all breakpoints regardless of the code. <h2>Setup</h2> With Visual Studio 2008 installed, there is one additional requirement for enabling JavaScript debugging: Internet Explorer must have script debugging enabled. This setting is available via the Tools | Internet Options menu within Internet Explorer. Once the Internet Options dialog box is open, select the Advanced tab and the Disable Script Debugging option is located in the Browsing section. It should not be checked so that it is enabled. Figure A shows the Disable Script Debugging option deselected. Figure A <h6>Figure A: Enabling script debugging in Internet Explorer</h6> With debugging feature enabled in Internet Explorer, you can start debugging your JavaScript code. <h2>Features</h2> The JavaScript debugger presents many standard debugging features. The following list provides a review of some of these features. <ul><li>Breakpoints: Breakpoints allow you to stop code execution at specific points within the source code. JavaScript breakpoints work just like C#/VB breakpoints. Breakpoints are set/cleared by clicking to the left of the code within the IDE; by using the Debug menu or by using the Breakpoint Context menu available by right-clicking on the code. In addition, you may create conditional breakpoints to allow you to define conditions for the breakpoint, as well as set up a macro to run or a message to print when the breakpoint is triggered. Breakpoints are viewable in the Watch window located by default in the lower left of Visual Studio 2008.</li><li>Call Stack: The Call Stack tab in the lower right of Visual Studio 2008 allows you to view what is being called when the <a href="http://ASP.NET">ASP.NET</a> page is loaded/running.</li><li>Locals: The Locals tab in the lower left of Visual Studio 2008 allows you to view the values of variables and objects during script execution. Also, you may change the contents of one of the local variables while the page is running.</li><li>Execute code: The Immediate Window tab in the lower right of Visual Studio 2008 allows you to execute JavaScript code on-the-fly during page execution.</li></ul> The Debug drop-down menu within Visual Studio 2008 provides a Windows submenu that allows you to define what windows to display while debugging. The examples in this article use Locals, Watch, Call Stack, and Immediate Window. The most used feature during debugging is often breakpoints because you can use them to stop code execution and examine the state of variables and objects at that point in the script. Breakpoints provide a number of features (which are available via the Debug drop-down menu during a debugging session) for their use. Code execution stops at each breakpoint — unless it is a conditional breakpoint whose condition has not been met. Once the code is halted, you may choose to continue one line at a time or until the next breakpoint is encountered. The debugger is actually enabled via the Debug menu in Visual Studio 2008. You go to Debug | Start to begin a debugging session or press [F5]. A picture is worth a thousand words, so let's take a closer look at actually using the debugger. <h2>In action</h2> For a simple example, I have an <a href="http://ASP.NET">ASP.NET</a> page that includes three JavaScript functions. One function populates a DIV tag on the page with text passed to it. The second function displays text passed to it in a pop-up window via the alert function. The last function populates a DIV tag with the text "Loaded"; this function is called when the page first loads. Figure B shows this page running in debug mode. A breakpoint has been set on the line that populates a local variable with the contents of a DIV tag on the page. The red circle to the left of the line lets you know that it is a breakpoint. The line of code waiting to run is highlighted in yellow. Figure B <h6>Figure B: Sample page opened for debugging in Visual Studio 2008</h6> Also, a watch has been set on the divText variable, so the many properties associated with it are displayed in the Watch window at the bottom of the screen. Watches have been greatly enhanced in Visual Studio 2008 to include much more detail. You can now view runtime object's methods and events, as well as property and property type information. <h2>Dynamic nature</h2> A unique feature of JavaScript is its dynamic nature, whereas the code actually loaded in the browser when an application runs is often different than what is in the IDE. That is, the code is often generated as scripts may be stored as resources, loaded from other locations, or contained within other compiled controls. The Script Documents feature of Visual Studio 2008 allows you to easily keep an eye on what is loaded and where it originated. While debugging, the Script Documents node will appear within the Solution Explorer pane. It will include the URL of each resource loaded into the page. It is visible in the upper right of Figure B. Another note on the dynamic nature of JavaScript is the setting of breakpoints in the source code. Visual Studio 2008 automatically maps the breakpoint location to the dynamically generated page loaded in the browser. Likewise, if you set breakpoints in the executing page, the IDE performs the reverse mapping to update the breakpoints in the source code. <h2>Take a peek</h2> Debugging is an essential aspect of every developer's job. Debugging standard code has never been a problem, but working with JavaScript has always presented problems. Visual Studio 2008 eases the burden with an enhanced debugger that simplifies the chore of taking a look at what your JavaScript is actually doing at runtime. What tools do you use to work with JavaScript? Do you plan on using Visual Studio 2008 to work with JavaScript now or in the future? Share your thoughts with the Web Developer community. <ul class="postData"><li>Date: May 15th, 2008</li><li>Author: Tony Patton </li></ul> <div class="blogger-post-footer"></div> </article> <article> <h1>Why it’s impossible to become a programming expert</h1> 2008-12-23T12:33:00.001+05:00 The pace of change in the technology industry has made it nearly impossible to specialize in much or become an expert in anything. I started cluing in on this a few years ago when I was reading a lot about Lisp, but I simply didn't have time to try it out. Learning Lisp, and learning it well, requires a lot of reading, practice, trial and error, and so on. This would have been fine and dandy, but I wasn't doing this during my 9-5 job; this was just a "wouldn't it be neat to learn this?" type of project. A few months ago, my father and I were discussing the topic of expertise, and some of the things he said really clicked in my head. Compare the amount of knowledge needed to really know C, Pascal, or maybe COBOL to their modern business programming analogues of J2EE and .NET. There are a few commands, a few primitive types, and a few operators. Let's look at the libraries in 1991 as an example (1991 is the year I learned to program). Windows was just starting to penetrate the enterprise. A lot of programming was happening on mainframes. By and large, UIs were completely text based without a mouse. Input validation was a cinch; you simply looped over a "wait for input" statement until the user pressed one of the three valid buttons for that point in the program. Events? Nope. Object-oriented programming? Nope. Declarative UI definitions (HTML, XAML, etc.)? Nope. N-tiered architecture? Accessing resources over a network? Globalization? Nope, nope, and nope. Life was pretty darned simple. It was a really nice transition from that world to the Perl/CGI world. At the end of the day, writing Perl in a CGI environment is conceptually very similar to writing a COBOL program: You take a batch of input, do your processing, and dump your output as a batch. It took roughly 50 printed pages to have enough Perl documentation, examples, "cookbook recipes," and so on to do the job at a decent level of competency. The experts were the folks who had been around long enough to understand issues such as SQL injection, and the need to properly escape encoded entities. At the time, I specialized in regular expressions. A few months of working with regular expressions (which are arguably a micro-language of their own), and I could read a regex like English. Over the next few years, I worked some real magic with them, from writing my own PHP-like system that fit my needs to working on screen scraping. In 2001, I transitioned to Java. All of a sudden, I had some monolithic 1,000 page tome on my desk and 200 MB of HTML with minimal formatting on my hard drive — and these were just as reference sources. I ended up reading 500-700 pages of a "teach yourself Java" type of book. (In comparison, I read <a target="_blank" href="http://portal.acm.org/citation.cfm?id=526615">C in Plain English</a> in three days — it was only a few hundred pages — without access to a C compiler, and I definitely qualified as "knowing" C.) The Java language has grown quite a bit since then; C# and <a href="http://VB.NET">VB.NET</a> are comparable in size and scope. I would guess that it would take at least 500 pages to adequately document (with examples and a thorough explanation of the techniques, usage scenarios for them, etc.) <a href="http://VB.NET">VB.NET</a>, C#, or Java. PHP is much less complex, thanks to its Perlish roots. We still haven't looked at the massive libraries that these systems carry with them; J2EE and the .NET Framework are both quite extensive. If anyone tells me that they are an expert in these systems, I know they are lying. I have been working with the .NET Framework since 2003, and I am not an expert in the Framework. I am highly experienced with a few namespaces: System.Drawing, System.Threading, System.Net.Web, and System.Text.RegularExpressions. The rest of my time working with .NET has been so insanely spread out that I can't remember much of it. Thank goodness for Visual Studio's IntelliSense; at least with that, I can muddle through namespaces until I find what I need. If anyone were to watch me write code, they would assume that I am an idiot, since it would definitely look like I was grasping at straws 50% of the time. To be honest, that is what it feels like. Unfortunately, my "grasping at straws 50% of the time" still beats the industry average of 70% (I'm just making up numbers). An expert programmer is no longer someone who is really knowledgeable or experienced. All too often, an expert programmer is the person who is adept at using a variety of reference tools and documentation to find out how to achieve their goals. This is my secret sauce. I am really good at looking at a problem, figuring out approximately what is wrong, and being able to quickly find the solution. To the uninitiated, it looks like sheer magic. To others, they say, "wow, Justin figured that out in only 30 minutes!" when they had been struggling with it all day. My real talent is knowing how to rapidly research and turn my findings into usable information. I suspect that I would be just as good at being a forensic accountant or a question writer for Jeopardy!. Most of the really good programmers I have met are the same way — they know a little of everything. They have tons of experiences that inform their "guiding light" when they look for answers. They have a natural talent, but overall, if you were to grill them on anything outside a narrow area (say, the System.Threading namespace), there is a really good chance that they will know where to get the answer from but not actually know the answer. Mark Cuban recently called this the <a target="_blank" href="http://www.blogmaverick.com/2008/03/10/we-live-in-an-open-book-world-the-lie-of-information-overload/">"Open Book" World</a>, and I tend to agree. Although for those of us in the IT industry, this happened 10 years ago; programmers (unlike network engineers and system administrators) have been able to touch radically new concepts with a simple download since about 1995. The eruption in new programming models, languages, frameworks, libraries, etc. closely tracks the adoption of the Internet. Between vendors and open source projects, it seems like another new "this will revolutionize how you program!" system is announced once a week or so. They all look worthy, but they take six months to learn really well, and I just can't devote my time to working with more than one at a time. So my choice is to either become barely familiar with a lot of things, or to commit myself to something that may not pan out. For now, the "sampler platter approach" has been working well for me at the professional level, although I find it quite frustrating at the personal level. I miss learning things in depth. I miss the sense of satisfaction from attaining a level of expertise. I miss getting to explore obtuse and obscure areas of knowledge. But it simply does not match the reality of my work or that of most other programmers. <ul class="postData"><li>Date: May 27th, 2008</li><li>Author: Justin James </li></ul> <div class="blogger-post-footer"></div> </article> <article> <h1>Manager won’t hire moonlighters</h1> 2008-12-23T12:32:00.001+05:00 <div class="entry"> A TechRepublic member has a problem with prospective employers who won't hire him because he has a side business. How should he handle it? ————————————————————————————————————— In a blog this week I wrote about how to convince a prospective employer that <a target="_blank" href="http://blogs.techrepublic.com.com/career/?p=340">you're not overqualified for a position</a>. One of our TechRepublic members was frustrated by the fact that a couple of the people who interviewed him for a developer position refused to believe that he would be happy stepping down from the management position he currently held. This week, another member wrote in with a similar complaint. In this case, prospective employers think some business endeavors he started while on a hiatus will interfere with any new full-time position he gets. Here's the scoop: "I took a year's hiatus from IT and started up a few small businesses (consulting, web development) to keep me busy and to learn some new skills. Now that I'm back to looking for a job, employers question the 'self-employed' item in my resume. I don't want to leave the last year blank and I don't want to lie about what I've been doing for the past year. I was told flat out at one interview that they were already having problems with two staff who had their own business on the side and weren't interested in taking on any more - end of interview!" My advice: How nice that this interviewer was so eager to make an assumption about your work ethic based on his experience with other people! Unfortunately, this is pretty common. Most of the time when an employer makes a bad assumption about a job candidate, it's usually because they've been burned by a similar situation before. It's not fair, but that's why it happens. The interviewer is assuming that you have money and sweat equity wrapped up in your companies, and that you'll surely want to keep them afloat. And, in his experience, that means you will probably use company time to make that happen. This is also a shame for him because he's missing out on a lot of good employees — a pretty good percentage of IT moonlight. I wonder how your side businesses are presented in your resume, and if there is any way to refer to the experience gained from the endeavors without referring to them as "side businesses." That is, instead of saying that from 2006-2007, you started a Web development business, maybe you could say that during that time period you consulted on Web developments issues. The latter speaks about your experience and expertise without implying that it is a business that requires attention that would take you away from a job. It may appear that you are withholding information, but you're actually just spinning it so the information will be more agreeable to anyone looking at it. It's a shame that the initiative that you showed during your time off is being held against you. I hope things work out, or at least you interview with a manager who hasn't been burned by freelancers before. <ul class="postData"><li>Date: June 27th, 2008</li><li>Author: Toni Bowers </li></ul> </div> <div class="blogger-post-footer"></div> </article> <article> <h1>AJAX should not mandate HTTP</h1> 2008-12-23T12:30:00.001+05:00 An interesting and unfortunate by-product of AJAX applications is that these applications rely upon the existence of an application server always being available. Many Web users are used to the idea that they can save a Web page to be read at their convenience. Even when there is Flash on the page, the page still works offline. After all, HTML is simply a document storage format, and HTTP is simply a protocol designed to optimally transmit HTML documents. But, in reality, more and more Web developers are assuming that the user will not want to save the Web page or lose network connectivity. I think that this is a mistake. A lot of frameworks have come out recently that are designed to tackle the offline problem; the frameworks' approach is usually to run a database on the client. The database can cache data from the "home server," and if the user makes changes while offline, that database can write the changes back to the "home server" when it becomes available again. These frameworks don't do anything about all of the back and forth requests to the app server that AJAX applications generate. Some developers try to tackle this problem by loading a miniature application server on the client via Java (or similar "applet" style technologies). This approach works. However, after a certain point, I wonder how many hoops someone needs to jump through to compensate for an inherent shortcoming of a particular technique before it becomes clear that a technique might not meet their needs. Even assuming that you can put a miniature application server and database server on the client that can cache and sync with the master servers whenever possible, you still have not solved the problem of "saving a Web page." The user doesn't think they are using an "application"; if it runs in a Web browser, they treat it like a Web page. Users are always clicking the Back button (which messes up AJAX apps) because they expect it to work like the Undo button. And incidentally, it does work like the Undo button in a more traditional step-by-step forms-based application. Ironically, one of the biggest issues that AJAX addresses in traditional Web development is the stateless nature of the HTTP protocol — and yet it suffers from its own lack of state. Let's take a "product finder" application as an example. In the application, the user selects various options as "must have" and "nice to have" features. Some of these are Boolean selections, while others might be numeric ranges, and one or two might be "one-of-many" or "many-of-many" options. Now, let's imagine that our user has narrowed her choices down to four products from the original 50. She decided to save the Web page to disk so she can e-mail it to her boss or coworkers for review. Whoops! Her boss and coworkers will likely have a problem if they try to bookmark the page or e-mail the link. In a traditional Web application, if the developer is smart, he or she will use GET rather than POST for these scenarios to ensure that the link accurately represents the application's state, and the page is quite easily saved in its current state. I am not saying that AJAX applications cannot function like this; however, until users get used to the idea that the traditional model no longer applies, developers will want to find smart and creative ways of addressing this issue. One way of doing this would be to periodically "tag" the state with a hash code and redirect the browser to a URL that calls that hash code (for example, <a href="http://www.site.com/application/j7qw132j">http://www.site.com/application/j7qw132j</a>). This way, if the user hits the Back button, she goes back, say, two minutes in time (or 10 operations) rather than back to the beginning. Also, the user can e-mail (or bookmark) that link. Developers often build in E-mail This Page, Bookmark This Page, or even Save This Page buttons on their site, but it takes effort to build in the buttons, and users often miss those buttons or assume that they do the same thing as the similarly labeled browser functions. I think that developers who work hard can find some creative, workable, and usable solutions for this issue, and their users will appreciate it. <ul class="postData"><li>Date: May 30th, 2008</li><li>Author: Justin James </li></ul> <div class="blogger-post-footer"></div> </article> <article> <h1>Cyber-bullying- the impact and the consequence</h1> 2008-12-23T12:27:00.001+05:00 This is a difficult story on many fronts. In reading the facts, it is hard to believe that this kind of thing can happen. There may be a desire to blame the victim- who should have been "smarter." There is also a desire to shake the perpetrator who clearly should have known better. But perhaps it is a cautionary tale that we can all learn something from on many levels- not only personal, but from a corporate standpoint as well. On May 15, 2008, a federal grand jury indicted Lori Drew, 49, of O'Fallon, Missouri with one count of conspiracy and three counts of unauthorized computer access. If convicted, each count could net Ms Drew five years. Why does this matter? Allegedly, Ms. Drew, her 13 year old daughter, and Ms Ashley Grillis, 18, who was employed by the Drew family, created a fictitious MySpace profile for the purpose of harassing Megan Meier, a teenaged girl living on the same block as Drew and her family. For the next four weeks, a flirtatious relationship blossomed between Ms Meier and the fictitious person that Drew and Grillis created. And then it suddenly ended with the fictitious person telling Megan "The world would be a better place without you." Megan hung herself in her bedroom about an hour later. State lawmakers gave final approval on May 17th to a bill intended to deal with cyber-harassment. This bill is expected to cover several aspects of technology, identifying as illegal the harassment from computers, text messages, and other electronic devices as well as standard written and telephone communications. From <a href="http://www.efluxmedia.com/news_MySpace_Suicide_Case_Leads_To_Tougher_Laws_17707.html" title="eFlux Media" target="_blank">eFlux Media</a>: <blockquote>Republican Governor Matt Blunt, who will soon sign the final version of the bill, issued a statement saying that the social networking sites and technology have provided criminals with a whole new set of weapons against their victims and the new protections and penalties are highly needed in order to ensure a safer environment for everyone.</blockquote> While this case is certainly tragic, it is thankfully not the norm in social networking- a phenomenon that is taking the web, and business, by storm. What it tells us is that there are definitely boundaries, and states are now recognizing those boundaries and stepping up to set laws in place to enforce those boundaries. Blogs, wikis, and other social networking tools are beginning to find their way into the business corporate culture. They are useful tools for communication and collaboration between business partners in both the internal and external corporate structure and will continue to grow. According to Forrester Research, the foundation technology, Web 2.0, will cost business US$4.6 billion over the next five years, making Web 2.0 hotter in terms of growth than the area of business intelligence. I can see that. Seems like everyone I meet wants to add me to one friend list or another. But as business embraces social networking, one thing is clear. They will have to embrace some policy definitions along with the tools to insure that employees understand where the lines are firmly drawn. Does your business use any kind of social networking tools? Do you have a community blog on your intranet or use a wiki to explain the workplace jargon? <a href="http://techrepublic.com.com/5208-6230-0.html?forumID=102&threadID=263884" title="TR Discussion" target="_blank">How does business set the standards for your communication?</a> <ul class="postData"><li>Date: May 19th, 2008</li><li>Author: Tricia Liebert </li></ul> <div class="blogger-post-footer"></div> </article> <article> <h1>Alpha Five: Data-driven application development done right</h1> 2008-12-23T12:26:00.001+05:00 <div class="entry"> <a target="_blank" href="http://www.alphasoftware.com/">Alpha Software</a> specializes in creating tools that let developers easily create data-driven applications. Richard Rabins (Co-Chairman) and his brother Selwyn Rabins (CTO) showed me the ropes on the latest version of its <a target="_blank" href="https://www.alphasoftware.com/shop/alphafive/v9/">Alpha Five</a> development package, and I'm pretty impressed. <h2>What is Alpha Five?</h2> Alpha Five is a package that allows developers to rapidly and easily create data-driven applications. I know… we've all heard this promise a dozen times before, so now we're pretty jaded by it. In this case, I am inclined to believe that someone got the formula right. From the demonstrations that I saw, Alpha Five "connects the dots" that most of the mainstream products just simply leave unconnected. For example, we're all familiar with the pain of putting together the standard master grid/detail view with a search box Web page. Here is a great example of the benefits of Alpha Five that I saw in action. Let's say that you need to put a grid component on the screen and bind it to a data view and a detail view component. So you bind the detail view to a data view and bind that data view to the selected item in the master data view. Next, you make a search area, add the validation, and add its information to the master data view's filter. It is a formulaic piece of code, yet we repeatedly keep writing it. The code usually takes about 30 minutes to create and tidy up — or even longer if you are not familiar with it. Alpha Five makes this much easier. Instead, you put a grid component on the form, check a box to indicate that you want a detail view, and then select the fields for the master data view and the detail view. For the search area, check off fields to be "lookup" and add the constraints on the values entered. In less than 60 seconds, I saw Richard put together a form that always takes me 30 minutes. In a nutshell, the folks at Alpha Software asked, "What are the top X tasks that developers do that involve gluing standard components together in the same way day in and day out?" Then they took that list, wrote their own version of the components, and made those standard "cookbook recipe" combinations into component properties. My first reaction when I watched Richard and Selwyn show this off was, "Wow, this makes it a snap to write that standard CRUD Web application that I keep being asked to write!" They also showed off the company's Script Genie; it reminds me a lot of Outlook's Rules Wizard, but it generates code. Script Genie looks pretty slick: You select the actions that you want to take in response to various events. Then, you provide the parameters to the actions by selecting the properties of the objects that would be in scope and are appropriate. For example, you can use Script Genie to hook onto a button being clicked, select the Send Email action, and tell it to get the subject line from a configuration file and the body from a field on the screen. Regular readers know that <a target="_blank" href="http://blogs.techrepublic.com.com/programming-and-development/?p=46">I'm not a huge fan of AJAX</a>. However, the AJAX libraries that Richard and Selwyn showed me seem to be high quality and worked as promised. Of course, you still need to cope with the usual AJAX problems (like constantly hitting the Web server, but at least Alpha Five uses persistent TCP/IP connections which helps); but if you need to use AJAX, Alpha Five's libraries look pretty good. <h2>The technical details</h2> Alpha Five can make desktop apps (that are interpreted through a runtime system, similar to the JVM or .NET CLR) or Web apps. (Alpha Software's application server can run side-by-side with IIS, and the company is working on a clustering application server, as well as a non-Windows version.) The Web system makes pages similar to PHP, ASP, or JSP, with the A5 tags embedded in between HTML. The language Alpha Software uses is XBASIC, a variant of BASIC. For database connectivity, the <a target="_blank" href="https://www.alphasoftware.com/shop/alphafive/v9/">Alpha Five Standard Edition</a> uses its own database engine; the <a target="_blank" href="https://www.alphasoftware.com/shop/alphafive/v9/">Alpha Five Platinum Edition</a> supports other databases. Like Cognos, Alpha Software's tools produce a "standardized SQL," which then gets translated to the appropriate SQL statements depending upon which SQL driver you use. (Note: You can use ODBC if you want, but Alpha Software has brewed its own drivers for databases that are faster since they don't have ODBC's overhead.) A while back, I came to realize that most data-driven applications can be done in Microsoft Access or FoxPro; there's a thin veneer of data validation and business logic on top of a 10 - 30 table database — nothing too complex. Tools like Ruby on Rails address this and so does Alpha Five. Richard admits that Alpha Five is not a "kitchen sink" system like J2EE or .NET. But from what I saw, it can handle about 80% of the applications most developers write with significantly less effort than J2EE, PHP, or .NET require. By focusing on the stuff that most developers spend all day on (instead of what a few developers spend a little bit of time on), the Alpha Software team has created a system that does exactly what most developers need. Watching Richard and Selwyn go through the motions of using it, programmers will still need to write code by hand, but they wouldn't need to write as much code. <h2>Data binding done right</h2> We all know how I usually feel about these systems. I like to throw phrases out there like "shake 'n bake programmers" and knock "drag 'n drop data binding." I'll be the first to admit that years of hearing vendors promise "systems so easy, your secretary will be writing her own applications!" that I just don't believe it. What I saw of Alpha Five wasn't an attempt to get the secretary to be writing Web applications, but it was something that lets a programmer leverage his or her time better. <ul class="postData"><li>Date: June 5th, 2008</li><li>Author: Justin James </li></ul> </div> <div class="blogger-post-footer"></div> </article> <article> <h1>Installing a simple Web server on Windows Server 2008 from a script</h1> 2008-12-23T11:05:00.001+05:00 <div class="entry"> While there are more advanced Web server configurations, many products require the IIS Web engine as a prerequisite. Here's how to install a simple IIS Web server through a scripted role. ——————————————————————————————————————- You can usually add features through the Windows Server 2008 Server Manager snap-in via a script. In the case of IIS 7.0, there are many options available for the feature install. One way to ensure a consistent configuration for applications that require IIS (e.g., SQL Server) is to use a scripted installation. For IIS, the package manager can install the features locally. To install a basic IIS configuration, run the following script: <pre>start /w pkgmgr /iu:IIS-WebServerRole;IIS-WebServerManagementTools; IIS-ManagementService;IIS-Metabase;WAS-WindowsActivationService; WAS-ProcessModel;WAS-NetFxEnvironment;WAS-ConfigurationAPI</pre> The command provides little interaction and, by default, does not provide feedback of any type if an incorrect parameter is passed after the /iu parameter. Figure A shows IIS being added via the script. Figure A Once the scripted task completes after a minute or so, and the Window Server 2008 Server Manager snap-in performs its next refresh, the Web server is listed as a role on the system and is running. Figure B shows an updated Server Manager snap-in with this basic IIS role. Figure B Using this scripted install of IIS is a good way to ensure consistent server build configurations, and other Windows features can be automated through the package manager (pkgmgr). The package manager can also uninstall packages, which may be helpful for temporarily enabling a feature for a specified amount of time and for removing the feature from a scheduled task to ensure its decommissioning. For more information on the package manager, read the TechNet article <a target="_blank" href="http://technet.microsoft.com/en-us/library/cc749465.aspx">Package Manager Command-Line Options</a>. Stay on top of the latest Windows Server 2003 and Windows Server 2008 tips and tricks with our free Windows Server newsletter, delivered each Wednesday. <a href="http://nl.com.com/MiniFormHandler?brand=techrepublic&list_id=e071">Automatically sign up today! </a> <ul class="postData"><li>Date: September 23rd, 2008</li> <li>Author: Rick Vanover </li></ul> </div> <div class="blogger-post-footer"></div> </article> <article> <h1>Phorm’s Webwise: It’s back and gaining traction</h1> 2008-12-23T11:02:00.001+05:00 It appears that the company <a target="_blank" href="http://www.phorm.com/" title="Phorm">Phorm</a> and its application <a target="_blank" href="http://www.webwise.com/" title="webwise">Webwise</a> are alive and well. Phorm and several major British ISPs are putting a new spin on Webwise and how it will make everyone's Internet experience better. For instance this is <a target="_blank" href="http://www2.bt.com/static/i/btretail/webwise/" title="BT take">British Telecommunications Group's</a> (BT) take on Webwise: <blockquote>"BT Webwise increases your protection against online fraud and makes ads that appear on participating websites more relevant to your interests. It's completely free for BT Total Broadband customers and you don't have to download or install any software for it to work. BT Webwise automatically adds an additional layer of protection against online fraud by checking the sites you visit against a list of suspected fraudulent and untrustworthy websites. When you attempt to visit any website on the list, you'll see a warning, so you can choose whether or not to visit it. It's another way BT is helping to protect you online."</blockquote> It's free? Webwise requires massive amounts of hardware and software to check every single bit of network traffic that passes through the ISP. So, if you're wondering why an ISP would offer this service for free, wonder no longer. BT explains that your advertising (hint hint) experience will be more personal: <blockquote>"BT Webwise also personalizes the online advertising you see when browsing on participating websites by linking ads to your interests. For example, if you search for a weekend trip to Paris or visit pages related to Paris, BT Webwise would replace the standard ads that would normally appear with advertising relating to travel or hotels information. You won't see any more adverts than you normally do — they'll simply be more relevant."</blockquote> The reason it's free to the ISP members is that Webwise will become a major revenue stream for the ISP. As I understand the process, advertisers will pay Phorm and Phorm will then pay the ISPs. So, the ISPs are hoping that members will go along with it. Some history Back in July of this year I wrote two articles about new technology that has the potential to track and shape everyone's Internet traffic. "<a target="_blank" href="http://blogs.techrepublic.com.com/networking/?p=609" title="DPI">Deep Packet Inspection: What You Need to Know</a>" discusses technology that enables real-time deep packet inspection (DPI) of traffic. DPI has allowed companies to develop behavioral targeting applications that can shape traffic and inject third-party vendor (TPV) advertisements. The article "<a target="_blank" href="http://blogs.techrepublic.com.com/networking/?p=612" title="Behavioral Targeting">Behavioral Targeting: What You Need to Know</a>" discusses one such company, Phorm, and its traffic-shaping application Webwise. Just to keep all of us on the same page, a high-level view of behavioral targeting might be helpful. Briefly, behavioral targeting first determines what you like, based on where you go on the Internet. Then, behavioral targeting selects advertisements that are most likely to influence you, displaying them on the new Web pages you ask for. Back to the infamous cookie yet again If your ISP uses Webwise, your browser is given a cookie from the Webwise Web site, even though the Webwise site was never visited. This cookie contains a unique identifying number (UID), which identifies you to the advertising network. Then every time you surf to a new Web site, the UID along with information about that Web site is captured by Webwise. The UID is then compared to a database of previously visited Web sites and information about your browsing habits. After which Webwise will return what it considers relevant advertising information to your Web browser. Phorm is supposedly making Webwise an opt-in option now, which appears to be satisfying some of the privacy advocates. The reason I say some is that Webwise still installs a UID cookie for every Web page that you visit, even if you have opted out. Webwise still has to monitor all your surfing as it's the only way the application can read the opted-out status of the cookie. Therein lays the crux of the matter — mission creep. The ISP and Phorm can potentially track your whole Internet experience. Since DPI is being used, the tracking and scanning of information isn't limited to Web browsing. E-mail and virtually any traffic of interest could be captured and analyzed. Which ISPs are involved? As of now <a target="_blank" href="http://www2.bt.com/static/i/btretail/webwise/index.html" title="BT">BT</a>, <a target="_blank" href="http://www.virginmedia.com/customers/webwise.php" title="Virgin Media">Virgin Media</a>, and <a target="_blank" href="http://webwise.phorm.com/talktalk_webwise_pilot.php" title="Talk Talk">TalkTalk</a> have conducted tests or are in the process of testing. All indications are that the ISPs will in the near future launch the Webwise program. Preventative measures There are options that you can use to avoid behavioral targeting cookies and DPI scrutiny. Encrypted tunnels through your ISP disallow the installation of behavioral targeting cookies. Also using VPNs, whether they are IPsec, L2TP, or SSL, will negate any effort by DPI to decipher the encrypted traffic. E-mail is another subject, and once again the only sure way to ensure its privacy is to encrypt the message. There are not a whole lot of options, but that's because behavioral targeting applications are being placed only one hop away from your network perimeter. Final thoughts Whether this technology gains traction or not is going to depend on the legality of it and whether people are comfortable with having their Internet experience monitored. It appears that the British government doesn't consider it a <a target="_blank" href="http://www.theregister.co.uk/2008/09/22/bt_phorm_police_drop/" title="privacy">privacy</a> or <a target="_blank" href="http://www.publications.parliament.uk/pa/ld200708/ldhansrd/text/81118w0002.htm" title="copyright">copyright</a> issue. It will be interesting to see if the new spin Phorm is placing on Webwise will be sufficient to overcome member concern about privacy issues. Regarding the members of the the three ISPs, I recently read the Register's article "<a target="_blank" href="http://www.theregister.co.uk/2008/11/19/bt_phorm_censor/" title="Censor">BT Silences Customers over Phorm</a>." One has to wonder about the logic behind that. I suspect it will be interpreted as BT having something to hide. Need help keeping systems connected and running at high efficiency? Delivered Monday and Wednesday, TechRepublic's Network Administrator newsletter has the tips and tricks you need to better configure, support, and optimize your network. <a target="_blank" href="http://nl.com.com/MiniFormHandler?brand=techrepublic&list_id=e102" title="sign up">Automatically sign up</a> today!<ul class="postData"><li>Date: December 22nd, 2008</li> <li>Author: Michael Kassner </li><li>Ref: <a href="http://blogs.techrepublic.com.com/networking/?p=768&tag=nl.e102">http://blogs.techrepublic.com.com/networking/?p=768&tag=nl.e102</a> </li></ul> <div class="blogger-post-footer"></div> </article> <article> <h1>Success in Ur Official Carrier Tips</h1> 2008-11-25T13:34:00.001+05:00 <a href="http://2.bp.blogspot.com/_Qi0BBZfnzVg/SSu4hnLiMGI/AAAAAAAAADY/yKPJiso5wn4/s1600-h/join__Friends-kingdom__01-753865.jpg"></a><a href="http://2.bp.blogspot.com/_Qi0BBZfnzVg/SSu4hm5uLEI/AAAAAAAAADg/_f-iPxf9KjI/s1600-h/join__Friends-kingdom__02-754387.jpg"></a><a href="http://4.bp.blogspot.com/_Qi0BBZfnzVg/SSu4hwSjWbI/AAAAAAAAADo/K5zw-86CdSs/s1600-h/join__Friends-kingdom__03-755875.jpg"></a><a href="http://1.bp.blogspot.com/_Qi0BBZfnzVg/SSu4ifrCamI/AAAAAAAAADw/svzscDMsYKI/s1600-h/join__Friends-kingdom__04-757337.jpg"></a><a href="http://3.bp.blogspot.com/_Qi0BBZfnzVg/SSu4ih5XuRI/AAAAAAAAAD4/Rs6EAvihy5M/s1600-h/join__Friends-kingdom__05-758067.jpg"></a><a href="http://1.bp.blogspot.com/_Qi0BBZfnzVg/SSu4iinR_lI/AAAAAAAAAEA/7CeeHUXn6Qc/s1600-h/join__Friends-kingdom__06-758878.jpg"></a><a href="http://4.bp.blogspot.com/_Qi0BBZfnzVg/SSu4i-mPQNI/AAAAAAAAAEI/S-85q3oA07o/s1600-h/join__Friends-kingdom__07-759253.jpg"></a><a href="http://4.bp.blogspot.com/_Qi0BBZfnzVg/SSu4i2T3hZI/AAAAAAAAAEQ/P8mT8Q5BeC8/s1600-h/join__Friends-kingdom__08-759779.jpg"></a><a href="http://1.bp.blogspot.com/_Qi0BBZfnzVg/SSu4jGNjcuI/AAAAAAAAAEY/LkKf0jkBHFc/s1600-h/join__Friends-kingdom__09-760836.jpg"></a><a href="http://1.bp.blogspot.com/_Qi0BBZfnzVg/SSu4jQD2frI/AAAAAAAAAEg/8oCsgWMDzKA/s1600-h/join__Friends-kingdom__10-761937.jpg"></a><a href="http://4.bp.blogspot.com/_Qi0BBZfnzVg/SSu4j5afnfI/AAAAAAAAAEo/UQUg5M5eQ3U/s1600-h/join__Friends-kingdom__11-763297.jpg"></a> <div class="blogger-post-footer"></div> </article> <article> <h1>Success in Ur Official Carrier__Tips</h1> 2008-11-25T11:07:00.002+05:00 <div class="gmail_quote"><div><div style="font-family: times new roman,new york,times,serif; font-size: 10pt;"><div style="font-family: times new roman,new york,times,serif; font-size: 10pt;"><div style="font-family: times new roman,new york,times,serif; font-size: 12pt;"> <a rel="nofollow" href="#" target="_blank"></a> <a rel="nofollow" href="#" target="_blank"></a> <a rel="nofollow" href="#" target="_blank"></a> <a rel="nofollow" href="#" target="_blank"></a> <a rel="nofollow" href="#" target="_blank"></a> <a rel="nofollow" href="#" target="_blank"></a> <a rel="nofollow" href="#" target="_blank"></a> <a rel="nofollow" href="#" target="_blank"></a> <a rel="nofollow" href="#" target="_blank"></a> <a rel="nofollow" href="#" target="_blank"></a> <a rel="nofollow" href="#" target="_blank"></a> </div></div></div> </div></div><div class="blogger-post-footer"></div> </article> <article> <h1>Use Cases writing Tips</h1> 2008-10-31T10:44:00.001+06:00 Use Case Definition: " A Use case us the specification of a set of actions performed by a system, which yields an observable result that is, typically, of value for one or more actors or other stake holders of the system (UML 2) " TIPS: <ul><li>A Use Case is the story of how the business or System and the user interact.</li><li>Use Cases are formal requirements that clearly define the resultant value.</li><li>Use Cases transform "shall" statements into groups that provide observable value and context, organized from a user prospective.</li> <li> Use Cases describe both functionality and results.</li><li>A common mistake is to consider requirements with design specifications.</li><li>Diagrams provide a visual reference for the use case.</li><li>People are the most important use case element.</li> <li>Establishing a use case style makes writing and reading them faster and easier.</li><li>Use Cases should have a single main flow and multiple alternative flows.</li><li>Alternative flows explain deviation from the main flow.</li> <li>Use Cases are not design documents they are requirement documents.</li><li>References define the beginning and the end of an alternative flow.</li><li>References indicate what caused an alternative flow to start and what the system does in response.</li> <li>Readability can suffer if there are "if statements" in a flow, because it usually means multiple requirements.</li><li>Selecting a single choice for an actor, and creating alternative flows for other choices, can simplify the main flow.</li> <li>A poorly written use case has too many create, read, update and delete activities, reducing clarity.</li><li>Removing CRUD creates a simpler document.</li><li>Sequencing the events in a flow is not always necessary to achieve clarity.</li> <li>Don't forget you are writing use cases for the end user. </li><li>Agreement on use cases structure and processes is essential to achieving quality and consistency.</li></ul> Ref: Tips for writing good use cases by James Heumann <div class="blogger-post-footer"></div> </article> <article> <h1>Rewriting the DotNetNuke Url Rewriter Module - Again</h1> 2008-09-16T13:59:00.001+06:00 <div dir="ltr">Ref: <a href="http://www.ifinity.com.au/Blog/Technical_Blog/EntryID/24/">http://www.ifinity.com.au/Blog/Technical_Blog/EntryID/24/</a> <div> <h2>Redoing the DotNetNuke Friendly Url Provider for Human Reading and Search Engines Indexing</h2> NOTE: Due to popular demand, there is now a <a href="http://www.ifinity.com.au/Products/Support_Forum/">Support Forum</a> for support requests on this module, or if you just want the thing to work ASAP, I can provide <a href="http://www.ifinity.com.au/Products/Premium_Support">one-on-one support or install it for you.</a> What I did previously was improve upon Scott McCulloch's work by working out a way to handle parameters. All well and good, and it worked OK. But it just still didn't look quite right. What's more, I wanted to use the rel="tag" microformat on my tagging module, and I was stuck with the .aspx extension on everything, which the microformat doesn't recognise. So after some back and forwards on the Ventrian forum where I posted my update, I went away and had a good think about it. There were two things nagging at me: <ul><li>1. The performance would drop off drastically with the number of pages in a site, due to the iterative search for a match.</li><li>2. The only way to get truly nice Url's is to ditch the page extension of .aspx. When you think about it, it doesn't serve the user at all. It's really only there to make it easier on the web server. It had to go</li></ul> Doing number 1 was easy : just do a dictionary based lookup on the page path, if it's found, great. If not, 404 coming right up. It works like this (if you're not intimately familiar with DNN, you might want to skip to the next part). On the first request, a dictionary of path information is built up. This contains the path (example: mysite/mypage) and the actual DNN request Url (example: default.aspx?tabid=37). The dictionary is stashed away in the Cache. So far so good. The incoming url is deconstructed into segments. Working backwards from the full url (mysite/mypage/mypath/myvalue) the url is tried against the dictionary for a 'hit'. If a page entry in the dictionary is found, great, the url is rewritten and processing continues. If the dictionary didn't contain the page, then the next segment is removed, thus mysite/mypage/mypath/myvalue is trimmed to mysite/mypage/mypath. This trimming process is repeated until there are not segments left. If nothing was found after all that, the page dictionary is rebuilt one more time, just in case it's a new page, then the process is repeated. If still nothing, then no url rewriting will be done and a 404 will probably occur. There is obviously a great deal more complexity in it than that, but that's the basic algorithm. So onto number 2. Some will look at you with fear in their eyes when you mention removing the .aspx extension from <a href="http://asp.net">asp.net</a> pages and declare that 'you cannot change the laws of physics'. But in reality there is nothing difficult about removing the .aspx extension from <a href="http://asp.net">asp.net</a> pages. There's really two (easy) ways. <ul><li>-> Implement an ISAPI Rewriting DLL. There's some commercial ones available, and some open-source ones. I assume these work OK, but looked like way too much work for me.</li><li>-> Direct all calls to the website through <a href="http://asp.net">asp.net</a> by mapping a wildcard (*) to the aspnet_isapi.dll</li></ul> Given I was already rewriting Urls, the second of those options was the path of least resistance for me. Now I realise that people using DotNetNuke on a shared host aren't going to be able to do this, but if you've got your own server, it's not difficult. By mapping all requests to go through <a href="http://asp.net">asp.net</a>, all requests for items on the website (or virtual directory) end up going through the DNN Url Rewriter. Eek - better make sure they work then. To fix this, I implemented a couple of regex filters to be placed in the web.config. These restrict what items will be passed along to IIS unfettered and what will be rewritten by the UrlRewrite code. And that was about it - because the way the dictionary lookup works, the .aspx is redundant anyway. Because all entries in the dictionary are without .aspx, it's not needed to find them again - only the relative path. Once the messy .aspx extensions are eradicated, getting nicer Url's is easy-peasy. Instead of doing by front-to-back flipping of parameters around, now I just put them out in-line, as they should. So, from the example given in the first posting on this topic, the results now are: <ul><li>Original,standard, DotNetNuke friendly Url: /MyPage/TabId/38/Key1/Value1/Key2/Value/default.aspx</li><li>My rewritten Url (first version): /MyPage/Value1/Key2/Value2/Key1.aspx</li> <li>My rewritten Url (no .aspx version): /MyPage/Key1/Value1/Key2/Value2</li></ul> Of course, there are still times when the first-parm-last scenario might work well, so I've left it in as a configurable option. This is particularly the case if you can't remove the .aspx extensions (shared hosting, for example) Here's the original examples from the first post redone: <ul><li>Directory Module <ul><li>Original: <a href="http://www.auctionlink.com.au/Auctioneers/TabId/54/Auctioneer/Grays_NSW/default.aspx">http://www.auctionlink.com.au/Auctioneers/TabId/54/Auctioneer/Grays_NSW/default.aspx</a></li> <li>First version: <a href="http://www.auctionlink.com.au/Auctioneers/Grays_NSW/Auctioneer.aspx">http://www.auctionlink.com.au/Auctioneers/Grays_NSW/Auctioneer.aspx</a></li><li>New Version: <a href="http://www.auctionlink.com.au/Auctioneers/Auctioneer/Grays_NSW/">http://www.auctionlink.com.au/Auctioneers/Auctioneer/Grays_NSW/</a></li> </ul> </li><li>Tagging Module <ul><li>Original: <a href="http://www.auctionlink.com.au/TagList/TabId/56/Tag/Auctioneers/default.aspx">http://www.auctionlink.com.au/TagList/TabId/56/Tag/Auctioneers/default.aspx</a></li> <li>First Version: <a href="http://www.auctionlink.com.au/TagList/Auctioneers/Tag.aspx">http://www.auctionlink.com.au/TagList/Auctioneers/Tag.aspx</a></li><li>New Version: <a href="http://www.auctionlink.com.au/TagList/Tag/Auctioneers">http://www.auctionlink.com.au/TagList/Tag/Auctioneers/</a></li> </ul> </li><li>Tagging Module with 2 parameters (tag name and page number) <ul><li>Original: <a href="http://www.auctionlink.com.au/TagList/TabId/56/Tag/Auctioneers/pg/2/default.aspx">http://www.auctionlink.com.au/TagList/TabId/56/Tag/Auctioneers/pg/2/default.aspx</a></li> <li>First Version: <a href="http://www.auctionlink.com.au/TagList/Auctioneers/pg/2/Tag.aspx">http://www.auctionlink.com.au/TagList/Auctioneers/pg/2/Tag.aspx</a></li><li>New Version: <a href="http://www.auctionlink.com.au/TagList/Tag/Auctioneers/pg/2/">http://www.auctionlink.com.au/TagList/Tag/Auctioneers/pg/2/</a></li> </ul> </li></ul> <h2>What about the 301 Redirects?</h2> The 301 redirect code in the original version was an important step forwards for my SEO DotNetNuke efforts. DNN has a habit of outputting a variety of Url's for the same bit of content, so it's important to stay on top of this by both generating a single Url per bit of content, but also to let Google know you've been on the case of hunting down every last default.aspx?tabid=37 and home/tabid/37/default.aspx reference. So the new code maintains the previous features, and adds a couple more - when a request comes in that doesn't end in a '/', it puts one on. If a request comes in for a page with .aspx, and you've turned .aspx off, then it returns a 301 status and gives the new, cleaner url. As well as this, if you've got a page you've deleted from your website, or it was only visible for a period of time, then requests for the no-longer-valid page will redirect to the home page of your portal with a 301. <h2>Full feature list of this Url Rewriter / Friendly Url Provider</h2> Here's a list of all the things that this version can do: <ul><li>Url's are generated as friendly Urls by all code which uses the standard NavigateUrl() call in DotNetNuke.</li> <li>Choice of page extension - .aspx or any other extension (such as .page)</li><li>Choice of using page extensions - options are "always", "never", "pageOnly". Always and never are self-explanatory, pageonly means only use an extension when the Url is for a page that contains no query parameters.</li><li>301 redirects for 'unfriendly' page requests. This can be on or off.</li><li>301 redirects to home page for deleted and expired pages</li> <li>A regex filter can be implemented to restrict 301 redirects for matching Urls</li><li>Individual pages can be restricted from 301 redirects by placing in a delimited list</li><li>Choice of two ways of handling parameters/query strings - ordered, in which the values are shown as consecutive levels in a path (key1/value1/key2/value2) and "firstparmlast" in which the first key value is placed last (value1/key2/value2/key2).</li><li>Choice of detecting duplicate portal alias/page path combinations. It is possible to have portal1/test as a page, and a portal which has portal1/test as an alias, thus duplicating the same path on a single DNN install.</li></ul> <h2>How to install the DNN Friendly Url Provider</h2> Step 1: Download the code from the <a href="http://www.ifinity.com.au/Products/Free_Downloads/">Free Downloads</a> page. Step 2: Backup your existing DotNetNuke.HttpModules.UrlRewrite.dll from your website/bin directory, and copy the new version in to the /bin directory. Step 3: Backup your existing web.config, and then replace the existing 'DNNFriendlyUrl' provider section with this one: <div style="padding-left: 10px;"> <div><add name="DNNFriendlyUrl" type="DotNetNuke.Services.Url.FriendlyUrl.DNNFriendlyUrlProvider, DotNetNuke.HttpModules.UrlRewrite" includePageName="true" regexMatch="[^\+a-zA-Z0-9 _-]" urlFormat="HumanFriendly" redirectUnfriendly="true" doNotRewriteRegex="(\.axd)|(/DesktopModules/)" doNotRedirect="DirectorySearchResults;" doNotRedirectRegex="[.]*(/logoff.aspx)" pageExtensionUsage="never" parameterHandling="ordered" ignoreFileTypesRegex="(\.gif)|(\.css)|(\.js)|(\.jpg)|(\.html)|(\.htm)" checkForDupUrls="true"/></div> </div> Step 4: Change any options to suit how you'd like your DNN installation to work. The full list is below. Step 5: Try it out! If you haven't already done so, you might want to take a look at the <a href="http://www.ifinity.com.au/Blog/TechnicalBlog/tabid/60/EntryID/12/Default.aspx">DotNetNuke Google Sitemap Generator</a> available in the <a href="http://www.ifinity.com.au/Whatwedo/FreeDownloads/tabid/388/Default.aspx">free downloads</a> section <h2>Changing Config Entries for Different Options</h2> <table border="1" cellpadding="3" cellspacing="1" width="497"> <tbody> <tr> <td valign="top" width="245"> pageExtensionUsage="never"* </td> <td valign="top"> pageExtensionUsage="always" </td> </tr> <tr style="height: 8.25pt;"> <td valign="top" width="245"> /Enquiries/ </td> <td valign="top"> /Enquiries.aspx </td> </tr> <tr style="height: 14.25pt;"> <td valign="top" width="245"> pageExtensionUsage="never" * parameterHandling="ordered" </td> <td valign="top"> pageExtensionUsage="always" parameterHandling="ordered" </td> </tr> <tr style="height: 21pt;"> <td valign="top" width="245"> /TagList/Tag/Valuers/ </td> <td valign="top"> /TagList/Tag/Valuers.aspx </td> </tr> <tr style="height: 21pt;"> <td valign="top" width="245"> pageExtensionUsage="never" * parameterHandling="firstparmlast" </td> <td valign="top"> pageExtensionUsage="always" parameterHandling="firstparmlast" </td> </tr> <tr style="height: 21pt;"> <td valign="top" width="245"> /TagList/Valuers/Tag/ </td> <td valign="top"> /TagList/Valuers/Tag.aspx </td> </tr> <tr style="height: 14.25pt;"> <td valign="top" width="245"> pageExtensionUsage="pageonly" * </td> <td valign="top"> pageExtensionUsage="always" pageExtension=".page" ** </td> </tr> <tr style="height: 8.25pt;"> <td valign="top" width="245"> /Enquiries.aspx </td> <td valign="top"> /Enquiries.page </td> </tr> <tr style="height: 14.25pt;"> <td valign="top" width="245"> pageExtensionUsage="pageonly" parameterHandling="ordered" </td> <td valign="top"> pageExtensionUsage="always" pageExtension=".page" ** parameterHandling="ordered" </td> </tr> <tr style="height: 21pt;"> <td valign="top" width="245"> /TagList/Tag/Valuers/ </td> <td valign="top"> /TagList/Tag/Valuers.page </td> </tr> <tr style="height: 21pt;"> <td valign="top" width="245"> pageExtensionUsage="pageonly" * parameterHandling="firstparmlast" </td> <td valign="top"> pageExtensionUsage="always" pageExtension=".page" ** parameterHandling="firstparmlast" </td> </tr> <tr style="height: 21pt;"> <td valign="top" width="245"> /TagList/Valuers/Tag/ </td> <td valign="top"> /TagList/Valuers/Tag.page </td> </tr> </tbody> </table> <h3>Complete list of web.config options</h3> <ul><li>urlFormat (HumanFriendly,omitted) - 'humanFriendly' when using tabid less paths. If omitted, will use standard DNN-style friendly Urls</li> <li>doNotRewriteRegex - regex string for excluding incoming requests. If the incoming request matches the regex, no rewriting will be attempted. This is used to handle exceptions, such as direct calls to pages/controls in the DesktopModules path.</li><li>checkForDupURls (true/false) - true means an exception will be logged if duplicate urls are found while building the tab dictionary. Duplicate url's are urls where the combination of portalAlias/tabpath from two different portals match. Preference is always given to the 'first' portal / tab path combo, so the second and suqsequent combos will never resolve properly.</li><li>redirectUnfriendly (true/false) - whether to issue 301 redirect status codes if an incoming url does not match the friendly url for that page</li> <li>doNotRedirect - semicolon delimited list of tab paths where a 301 redirect should never be issued. Use when a particular page should use ?key=val type parameters, or unwanted results are occuring. This is only used when redirectUnfriendly = true</li><li>doNotRedirectRegex - a regex expression which, when evaluated against the incoming url, will not redirect if the result is a match. In the example, the /logoff.aspx page will never 301 redirect. This is only used when redirectUnfriendly = true </li><li>parameterHandling (ordered, firstlast) - ordered means key/value pairs are kept in order, and placed after the tab path. ie mypage/mykey1/myvalue1/mykey2/myvalue2. firstlast means take the key of the first parameter and place it last, at the end of the string. mypage/myvalue1/mykey2/myvalue2/mykey1. Default : firstlast</li><li>pageExtensionUsage (never, pageonly or always) - three options on whether to use page extensions (ie .aspx) for the pages. never = don't use page extensions, pageonly = only use page extensions when the page path does not include parameter key/value pairs (ie mysite/mypage.aspx but not mysite/mypage/mykey/myvalue), always = .aspx is appended to all urls. Default : alwaysUse. If using pageonly or never, the iis setup must have wildcard mapping to the aspnet_isapi.dll (instructions below)</li><li>ignoreFileTypeRegex (Regex string) - when mapping wildcard requests to aspnet_isapi.dll, the <a href="http://asp.net">asp.net</a> url rewriting function will be called for all types of iis resources on the page. This means unnecessary file handling for jpeg, gif, css,axd and other file types. By entering a regex string, any match found will bypass the url rewriting code and be passed back to iis.</li><li>pageExtension - allows the use of a page extension other than .aspx if required. (ie .page, .content - whatever)</li></ul> <h3>Changing IIS Settings for either a custom page extension, or no page extension</h3> Warning: This may destabilise your website. Always check this on a test version first, and understand what you are doing! These instructions will vary between IIS 5.0, IIS 6.0 and the Workstation and Server versions of 2000/XP/2003/Vista <h3>Changing IIS Settings to map all requests through the aspnet isapi dll</h3> <ul><li>open property page for website / virtual directory.</li><li>click the 'configuration' button, select the 'mappings' tab</li> <li>click on the 'add' button</li><li>Enter the details: <ul><li>Executable = \framework\\aspnet_isapi.dll</li><li>For no page extension -> Extension = .*</li><li>For custom page extension -> Extension = .{your value}</li> <li>Verbs = GET,POST,HEAD,DEBUG</li><li>Script Engine :checked</li><li>Check that file exists : unchecked</li></ul> </li><li>Click OK, OK, OK to close and apply changes.</li></ul> Then thoroughly test all website functions. And you're done! </div> <h2>Update (22 Nov 2007)</h2> Due to the problems with the 4.7 release and namespace collision with the config part of the module, I've released a new version which uses it's own namespace and should solve the problems. However, this requires new web.config entries. The changes are available in the free downloads section. This new version requires different modifications to the web.config from those shown in this blog post. The options are the same, but the namespace has changed. Please see the example.web.config file included in the downloads to see what changes are required for the web.config There are also three new features in this new version, which are set using the following configuration attributes: <ul><li>redirectWrongCase - allows 301 redirection of mixed case requests to lower case requests</li> <li>forceLowerCase - when true, forces all generated Urls to be in all lower case</li><li>redirectToSubDomain - when implemented, all requests will be 301 redirected to the specified subdomain, regardless of what subdomain they were requested on. Can be left as an empty string to remove the subdomain completely.</li></ul> Again, see the example.web.config on the different options for these features Any problems please report them using the comments section below (I might have to stretch to forums one day!) <h2>Update (10 Jan 2008)</h2> In response to the many requests to include a way of substituting spaces with a '-' or '_', I have included a 'replaceSpaceWith' option. This will replace any spaces in a tab path with the supplied character. It will also issue a 301 redirect for any request to the 'space removed' version. This means that existing pages found in search engines will redirect to the new version. I've also fixed (finally) the problems with getting an exception on the Host Settings page. Basically, in versions of DotNetNuke prior to 4.5, the Friendly Url Provider shared a namespace with internal classes. I've explictly implemented a cast so that my provider will be able to interact with the host settings page. However, for DNN versions 4.6 and above, the FriendlyUrl provider was moved into the single DotNetNuke.HttpModules assembly. This solved the problem, so the later version of my Friendly Url provider just doesn't include the section for reading the FriendlyUrl rules. The new files can be found on the Free Downloads page. You must install the correct version for your DNN install. If you are running 4.0 - 4.5, then you want the 4.5 version. If you are running 4.6 or later, then you want the 4.6 version. Trying to run the later version with the earlier DNN installs will just cause errors - there is no benefit in running the 4.6 code - it's just a compatibility change, both version are built from the same base source. Copyright Bruce Chapman 2007 </div> <div class="blogger-post-footer"></div> </article> <article> <h1>Rewriting the DotNetNuke Url Rewriter Module</h1> 2008-09-16T13:58:00.001+06:00 <div dir="ltr">Ref: <a href="http://www.ifinity.com.au/Blog/Technical_Blog/EntryID/19/">http://www.ifinity.com.au/Blog/Technical_Blog/EntryID/19/</a> <div> As part of my ongoing interest in making DotNetNuke websites more person and search engine friendly, I started hunting around in the space again to see what was available. While the latest DotNetNuke releases have good ability to enter site rewrite rules (such as an automatic home.aspx redirection), the crucial problem for me is that the site doesn't generate simple Url's (instead it still outputs pagename/tabid/nn/default.aspx) I had previously looked at the Inventua Http module, and while it is a good product, it didn't quite suit my needs. I guess it came down to not being able to get the source code either. It also didn't work with the Catalook module, which I have done some work with and still support on some sites. Further searching brought me across Scott McCulloch's 'Friendly Urls' work on his site at <a href="http://www.ventrian.com/Resources/Projects/FriendlyUrls.aspx">http://www.ventrian.com/Resources/Projects/FriendlyUrls.aspx</a>. (it's even a friendly url!) This showed a bit more promise, because he was automatically rewriting the Url's based on the path of the page. But Scott left 'Human Readable Urls for Pages With Multiple Parameters' listed under 'items for discussion'. <h2>How the Url Rewriting in DotNetNuke works</h2> Without going into the explicitly technical details, here's a quick primer. There are two facets to Url Rewriting in DotNetNuke: <ul><li>Generating Friendly Urls for Hyperlinks, menu items and other outputs.</li><li>Interpreting those Friendly Urls when requested, and working out which page to show.</li></ul> Generating Friendly Urls for Hyperlinks This is really the easy part, because theoretically you can generate the friendliest Url's in the world to show - it's only when they have to be re-interpreted that it becomes a problem! Having said that, this is how it works: <ul><li>Some code, somewhere calls DotNetNuke.Common.Globals.NavigateUrl(). This call requires the desired Tab (page), Portal, Url parameters if any. There's a few different versions of it, but they all distill down to the same call.</li><li>NavigateUrl determines the Http Alias of your Portal (ie <a href="http://www.ifinity.com.au">www.ifinity.com.au</a>), determines the TabId, and adds it to any query parameters. This results in the familiar Url of <a href="http://www.ifinity.com.au/default.aspx?tabid=38">www.ifinity.com.au/default.aspx?tabid=38</a></li><li>NavigateUrl then checks if the Host setting of 'UseFriendlyUrls' is set to 'true'. And if so, calls the DNNFriendlyUrl provider as specified in the web.config. By default this is 'DotNetNuke.HttpModules.UrlRewrite.dll', which is part of the base code.</li><li>The DNNFriendlyUrl provider must have an implementation of a function called 'FriendlyUrl', which reformats the generated Url into a 'friendly' format. In the case of the standard provider, the url of <a href="http://www.ifinity.com.au/default.aspx?tabid=38">www.ifinity.com.au/default.aspx?tabid=38</a> would come back as <a href="http://www.ifinity.com.au/Home/Tabid/38/default.aspx">www.ifinity.com.au/Home/Tabid/38/default.aspx</a></li></ul> The 'standard' FriendlyUrl provider does this by performing a simple reshuffling of the various parts of the querystring to achieve the necessary output. It's fast and works pretty well. Interpreting Friendly Urls The trickier part is determining what /home/tabid/38/default.aspx means when someone clicks on a hyperlink with this address. To do this, DotNetNuke uses the 'UrlRewrite' HttpModule. Now, in the standard DotNetNuke build, the UrlRewrite module and the FriendlyUrl provider are in the same assembly for ease of packaging and coding, but they don't necessarily have to be like that. When a Http Request comes into the DotNetNuke code (ie, you click on a hyperlink to load a new page) the DNN base throws the incoming Url to the UrlRewrite module, with an implicit request 'turn this into something I can understand, will you?' You should remember that the DNN base is just a page called default.aspx, and it only knows how to interpret a classic style query string - ie ?tabid=38&key1=value1&key2=value2 etc. This is the sequence of events: <ul><li>The incoming System.Web.HttpContext.Current.Request (Request for short) is directed to the UrlRewrite HttpModule, in the 'OnBeginRequest' event.</li><li>The request is inspected for certain exception cases, and then put through a Regex expression to determine the tabid and (sometimes) the portalId</li><li>Assuming the TabId is found in the incoming Url, the Request.Url is rewritten in the style expected, putting the TabId into the output parameters.</li><li>The request passes back to the DotNetNuke framework, and the requested page is loaded.</li></ul> All this happens for each and every request made to a DNN portal. So performance and scalability is paramount in any changes made in this area. If you are so inclined, I suggest stepping through the code one day just to see how often it gets called, and how much work goes on in the background to provide friendly Url's. It's when you do this you start to understand some of the issues surrounding a fully-dynamic website and fully-dynamic Url's. <h2>How I set out to Improve it for my own purposes</h2> Please note : the changes I am discussing here were changes I made to suit my own needs, and those needs are probably not aligned with many people in the DotNetNuke user group. So none of this is a criticism of the base code, or other people's work. It's just a discussion on getting it to work the way I wanted. My requirements were the same as what Scott outlined in his article: better Urls for human and search engine purposes. However, most work I do with DNN modules tends to involve a lot of parameters in the query string, and so his open problem of what to do with multiple querystring parameters was the same as my own. Specifically, I am developing two new modules: a Tagging module for tagging DNN content, and a Directory Module, for storage of Organization Directories. One of the principal aims of these two modules was nice-looking Urls with high keyword ratios in the Url. But I soon struck the same problem - what do you do with multiple parameters. My Answer My solution to this question was to re-arrange the order of the parameters, and use the key name of the first parameter as the page name in the Url. Confused? I bet! Here's what I mean: <ul><li>Original,standard, DotNetNuke friendly Url: /MyPage/TabId/38/Key1/Value1/Key2/Value/default.aspx</li><li>My rewritten Url: /MyPage/Value1/Key2/Value2/Key1.aspx</li></ul> While that may look a little strange and have you scratching your head, I can assure there's method to my madness. The first site I have implemented this on is a <a href="http://www.auctionlink.com.au/">List of Auctioneers in Australia</a>. It has the implementation of my new Tagging module and my new Directory module. Here are two Url's in that site, in 'original' and rewritten form: <ul><li>Directory Module <ul><li>Original: <a href="http://www.auctionlink.com.au/Auctioneers/TabId/54/Auctioneer/Grays_NSW/default.aspx">http://www.auctionlink.com.au/Auctioneers/TabId/54/Auctioneer/Grays_NSW/default.aspx</a></li><li>Rewritten: <a href="http://www.auctionlink.com.au/Auctioneers/Grays_NSW/Auctioneer.aspx">http://www.auctionlink.com.au/Auctioneers/Grays_NSW/Auctioneer.aspx</a></li> </ul> </li><li>Tagging Module <ul><li>Original: <a href="http://www.auctionlink.com.au/TagList/TabId/56/Tag/Auctioneers/default.aspx">http://www.auctionlink.com.au/TagList/TabId/56/Tag/Auctioneers/default.aspx</a></li><li>Rewritten: <a href="http://www.auctionlink.com.au/TagList/Auctioneers/Tag.aspx">http://www.auctionlink.com.au/TagList/Auctioneers/Tag.aspx</a></li> </ul> </li><li>Tagging Module with 2 parameters (tag name and page number) <ul><li>Original: <a href="http://www.auctionlink.com.au/TagList/TabId/56/Tag/Auctioneers/pg/2/default.aspx">http://www.auctionlink.com.au/TagList/TabId/56/Tag/Auctioneers/pg/2/default.aspx</a></li><li>Rewritten: <a href="http://www.auctionlink.com.au/TagList/Auctioneers/pg/2/Tag.aspx">http://www.auctionlink.com.au/TagList/Auctioneers/pg/2/Tag.aspx</a></li> </ul> </li></ul> Note: the rewriting has nothing to do with the module code, it is all in the HttpModule.URLRewrite Assembly. I consider this a 'first draft' because I'm still not totally happy with it. I actually want to implement the Tag miniformat, and to do that I need to have the tag url looking like this : /TagList/Tag/Auctioneer - with no parameters on the end. It's certainly possible given the code I've written, but I'll leave it for a bit further down the track. <h2>What about the old code?</h2> That all works great for a new site such as the <a href="http://www.auctionlink.com.au">www.auctionlink.com.au</a> site, as, in combination with my DotNetNuke Google Sitemap Provider, the googlebot has hoovered up all the friendly Url's and integrated them into the index nicely. So if you search for <a href="http://AuctionLink.com.au">AuctionLink.com.au</a> on Google, you should see the friendly Urls rather than the tabid/nn/default.aspx-style Urls. But what about sites that have been in the index for a while? Indeed, this is the problem with the <a href="http://ifinity.com.au">ifinity.com.au</a> site - it is in the Google index with the standard, friendly Urls. I'd like to implement my version of the UrlRewrite module for this site as well, and get some friendlier Url's happening. But Google and others have the old Url's already in the index, and, because the DNN framework will still respond to the older style Url's, I could even get search-engine penalised for duplicate content - which is when the search engines deem you to have two distinct Url's pointing to the same content. Even if that's not a problem, I'd still like to have the best Url's showing in the index. Enter the 301 Redirect After reading Matt Cutt's blog on Google about 301 redirects, it got me thinking- I could issue a 301 redirect for every request that came into the site for an older style Url. That way search engines which take notice of the Http standards should eventually update their indexes to include the new content. If you don't know what a 301 redirect is, it's a Http status code of '301 - Moved Permanently'. It's basically saying, yes I have the content for you, but now it's over here: please update your references. In some agents (certain browsers) it will actually call the new Url instead, where some agents will just ignore it and show the content if it's available. Google have stated (via Matt Cutt's blog) that they do read and obey 301, and you can use it to advice the Googlebot of new locations. <h2>Implementing 301 Redirects in UrlRewrite</h2> The code does this by detecting if a Url came into the site as an old style friendly Url - ie tabid/38/default.aspx. Then it works out what the 'friendly' url should be : home.aspx. If the incoming Url and the Friendly Url are different, then a 301 status is returned and the new friendly Url is given as the new location. There are exceptions to this, and there is a switch in the web.config to disable it once there are few 'old style' url's coming into the site (presumably once the search indexes are updated) There is also a section in the web.config for excluding certain pages from rewrites. For instance, in the auctionLINK site, I have implemented a search function that deliberately uses a non-friendly query string. This page is excluded from 301 redirects because I don't want the query string to be friendly. Here is the result, as shown by Fiddler, the free http monitoring tool distributed by Microsoft: <ul><li>Original Request (raw): GET /Home/tabid/37/Default.aspx HTTP/1.1 Accept: */* Accept-Language: en-au Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; InfoPath.1) Host: <a href="http://www.auctionlink.com.au">www.auctionlink.com.au</a> Proxy-Connection: Keep-Alive Pragma: no-cache</li><li>Response from Website (raw): HTTP/1.1 301 Moved Permanently Date: Thu, 30 Aug 2007 07:26:32 GMT Server: Microsoft-IIS/6.0 X-AspNet-Version: 2.0.50727 Location: <a href="http://www.auctionlink.com.au/Home.aspx">http://www.auctionlink.com.au/Home.aspx</a> Cache-Control: private Content-Length: 0 Connection: close</li></ul> <h2>Installing and Using the Code</h2> I have provided the source code for the HttpModule.UrlRewrite version in the <a href="http://www.ifinity.com.au/Blog/Technical_Blog/EntryID/19/">Free Downloads</a> page of this site. Steps to use it are: <ul><li>Download the code from the Free Downloads Page.</li><li>Copy the 'DotNetNuke.HttpModule.UrlRewrite' binary into your website's \bin directory (it won't do anything until you change the web.config)</li><li>Backup your web.config, then modify the 'DNNFriendlyUrl' provider section so that it looks like the following: </li><li>If you would prefer not to use the 301 redirects, set redirectUnfriendly="false"</li><li>If you would like to use the 301 redirects, but not for a certain page/pages, put the 'TabPath' value for those pages in a semi-colon delimited list - ie home;enquires;products/myproduct; (you should only use one forward slash, not two like the way it is stored in the database) Note in the example above, I have excluded 'SearchResults' from redirects.</li><li>If you are using Redirects, it's a good idea to use a tool like Fiddler to check that your site is working as expected. Check every page, because unfortunately it is possible for the redirect code to get stuck in a terminal loop (if anyone has an idea how to detect/stop this, I'm all ears)</li></ul> <h2>The Disclaimer</h2> This code is really in a BETA state. It is pretty fresh off the code production line and hasn't been totally tested in anger yet. If you do install it, please promise me that you will test it in a non-critical environment first, and that you will check every Url on your site to make sure that it works as you expect. I'm still developing it myself and may post updates if I think they are worthwhile. This code is a branch of Scott McCulloch's work, so full credit to him for getting me started in the right direction. And his work is a branch off the original work for the DotNetNuke base by Charles Nurse, so full credit to him as well. I don't claim credit for much of it at all, but by downloading it you are subjecting yourselef to the license of the DotNetNuke framework, so play nicely and attribute credit where credit is due. </div> Copyright Bruce Chapman 2007 </div> <div class="blogger-post-footer"></div> </article> <article> 2008-08-13T11:22:00.000+06:00 <param name="movie" value="http://i.netcarshow.com/ncs_show.swf?c=BMW&m=2008-M-Zero_Concept"><param name="wmode" value="transparent"></object><div class="blogger-post-footer"></div> </article> <article> <h1>Locking in Microsoft SQL Server</h1> 2008-07-22T18:47:00.001+06:00 <div dir="ltr"><h1>Introduction</h1>In this article, I want to tell you about SQL Server 7.0/2000 Transaction Isolation Levels, what kinds of Transaction Isolation Levels exist, and how you can set the appropriate Transaction Isolation Level, about Lock types and Locking optimizer hints, about deadlocks, and about how you can view locks by using the sp_lock stored procedure. <h2>Transaction Isolation Levels</h2>There are four isolation levels: <li>READ UNCOMMITTED <li>READ COMMITTED <li>REPEATABLE READ <li>SERIALIZABLE Microsoft SQL Server supports all of these Transaction Isolation Levels and can separate REPEATABLE READ and SERIALIZABLE. Let me to describe each isolation level. <h3>READ UNCOMMITTED</h3>When it's used, SQL Server not issue shared locks while reading data. So, you can read an uncommitted transaction that might get rolled back later. This isolation level is also called dirty read. This is the lowest isolation level. It ensures only that a physically corrupt data will not be read. <h3>READ COMMITTED</h3>This is the default isolation level in SQL Server. When it's used, SQL Server will use shared locks while reading data. It ensures that a physically corrupt data will not be read and will never read data that another application has changed and not yet committed, but it not ensures that the data will not be changed before the end of the transaction. <h3>REPEATABLE READ</h3>When it's used, the dirty reads and nonrepeatable reads cannot occur. It means that locks will be placed on all data that is used in a query, and another transactions cannot update the data. This is the definition of nonrepeatable read from SQL Server Books Online: <table width="100%"> <tbody> <tr bgcolor="#dcdcdc"> <td><pre>nonrepeatable read When a transaction reads the same row more than one time, and between the two (or more) reads, a separate transaction modifies that row. Because the row was modified between reads within the same transaction, each read produces different values, which introduces inconsistency. </pre></td></tr></tbody></table><pre></pre> <h3>SERIALIZABLE</h3>Most restrictive isolation level. When it's used, the phantom values cannot occur. It prevents other users from updating or inserting rows into the data set until the transaction will be completed. This is the definition of phantom from SQL Server Books Online: <table width="100%"> <tbody> <tr bgcolor="#dcdcdc"> <td><pre>phantom Phantom behavior occurs when a transaction attempts to select a row that does not exist and a second transaction inserts the row before the first transaction finishes. If the row is inserted, the row appears as a phantom to the first transaction, inconsistently appearing and disappearing. </pre></td></tr></tbody></table><pre></pre>You can set the appropriate isolation level for an entire SQL Server session by using the SET TRANSACTION ISOLATION LEVEL statement. This is the syntax from SQL Server Books Online: <table width="100%"> <tbody> <tr bgcolor="#dcdcdc"> <td><pre>SET TRANSACTION ISOLATION LEVEL { READ COMMITTED | READ UNCOMMITTED | REPEATABLE READ | SERIALIZABLE } </pre></td></tr></tbody></table><pre></pre>You can use the DBCC USEROPTIONS statement to determine the Transaction Isolation Level currently set. This command returns the set options that are active for the current connection. This is the example: <table width="100%"> <tbody> <tr bgcolor="#dcdcdc"> <td><pre>SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED GO DBCC USEROPTIONS GO </pre></td></tr></tbody></table><pre></pre> <h2><a name="part_3"></a>Lock types</h2>There are three main types of locks that SQL Server 7.0/2000 uses: <li>Shared locks <li>Update locks <li>Exclusive locks Shared locks are used for operations that do not change or update data, such as a SELECT statement. Update locks are used when SQL Server intends to modify a page, and later promotes the update page lock to an exclusive page lock before actually making the changes. Exclusive locks are used for the data modification operations, such as UPDATE, INSERT, or DELETE. Shared locks are compatible with other Shared locks or Update locks. Update locks are compatible with Shared locks only. Exclusive locks are not compatible with other lock types. Let me to describe it on the real example. There are four processes, which attempt to lock the same page of the same table. These processes start one after another, so Process1 is the first process, Process2 is the second process and so on. Process1 : SELECT Process2 : SELECT Process3 : UPDATE Process4 : SELECT Process1 sets the Shared lock on the page, because there are no another locks on this page. Process2 sets the Shared lock on the page, because Shared locks are compatible with other Shared locks. Process3 wants to modify data and wants to set Exclusive lock, but it cannot make it before Process1 and Process2 will be finished, because Exclusive lock is not compatible with other lock types. So, Process3 sets Update lock. Process4 cannot set Shared lock on the page before Process3 will be finished. So, there is no Lock starvation. Lock starvation occurs when read transactions can monopolize a table or page, forcing a write transaction to wait indefinitely. So, Process4 waits before Process3 will be finished. After Process1 and Process2 were finished, Process3 transfer Update lock into Exclusive lock to modify data. After Process3 was finished, Process4 sets the Shared lock on the page to select data. <h2><a name="part_4"></a>Locking optimizer hints</h2>SQL Server 7.0/2000 supports the following Locking optimizer hints: <li>NOLOCK <li>HOLDLOCK <li>UPDLOCK <li>TABLOCK <li>PAGLOCK <li>TABLOCKX <li>READCOMMITTED <li>READUNCOMMITTED <li>REPEATABLEREAD <li>SERIALIZABLE <li>READPAST <li>ROWLOCK NOLOCK is also known as "dirty reads". This option directs SQL Server not to issue shared locks and not to honor exclusive locks. So, if this option is specified, it is possible to read an uncommitted transaction. This results in higher concurrency and in lower consistency. HOLDLOCK directs SQL Server to hold a shared lock until completion of the transaction in which HOLDLOCK is used. You cannot use HOLDLOCK in a SELECT statement that includes the FOR BROWSE option. HOLDLOCK is equivalent to SERIALIZABLE. UPDLOCK instructs SQL Server to use update locks instead of shared locks while reading a table and holds them until the end of the command or transaction. TABLOCK takes a shared lock on the table that is held until the end of the command. If you also specify HOLDLOCK, the lock is held until the end of the transaction. PAGLOCK is used by default. Directs SQL Server to use shared page locks. TABLOCKX takes an exclusive lock on the table that is held until the end of the command or transaction. READCOMMITTED Perform a scan with the same locking semantics as a transaction running at the READ COMMITTED isolation level. By default, SQL Server operates at this isolation level. READUNCOMMITTED Equivalent to NOLOCK. REPEATABLEREAD Perform a scan with the same locking semantics as a transaction running at the REPEATABLE READ isolation level. SERIALIZABLE Perform a scan with the same locking semantics as a transaction running at the SERIALIZABLE isolation level. Equivalent to HOLDLOCK. READPAST Skip locked rows. This option causes a transaction to skip over rows locked by other transactions that would ordinarily appear in the result set, rather than block the transaction waiting for the other transactions to release their locks on these rows. The READPAST lock hint applies only to transactions operating at READ COMMITTED isolation and will read only past row-level locks. Applies only to the SELECT statement. You can only specify the READPAST lock in the READ COMMITTED or REPEATABLE READ isolation levels. ROWLOCK Use row-level locks rather than use the coarser-grained page- and table-level locks. You can specify one of these locking options in a SELECT statement. This is the example: SELECT au_fname FROM pubs..authors (holdlock) <h2><a name="part_5"></a>Deadlocks</h2>Deadlock occurs when two users have locks on separate objects and each user wants a lock on the other's object. For example, User1 has a lock on object "A" and wants a lock on object "B" and User2 has a lock on object "B" and wants a lock on object "A". In this case, SQL Server ends a deadlock by choosing the user, who will be a deadlock victim. After that, SQL Server rolls back the breaking user's transaction, sends message number 1205 to notify the user's application about breaking, and then allows the nonbreaking user's process to continue. You can decide which connection will be the candidate for deadlock victim by using SET DEADLOCK_PRIORITY. In other case, SQL Server selects the deadlock victim by choosing the process that completes the circular chain of locks. So, in a multiuser situation, your application should check the error 1205 to indicate that the transaction was rolled back, and if it's so, restart the transaction. Note. To reduce the chance of a deadlock, you should minimize the size of transactions and transaction times. <h2><a name="part_6"></a>View locks (sp_lock)</h2>Sometimes you need a reference to information about locks. Microsoft recommends using the sp_lock system stored procedure to report locks information. This very useful procedure returns the information about SQL Server process ID, which lock the data, about locked database, about locked table ID, about locked page and about type of locking (locktype column). This is the example of using the sp_lock system stored procedure: <table width="100%"> <tbody> <tr bgcolor="#dcdcdc"> <td><pre>spid locktype table_id page dbname ------ ----------------------------------- ----------- ----------- --------------- 11 Sh_intent 688005482 0 master 11 Ex_extent 0 336 tempdb </pre></td></tr></tbody></table><pre></pre>The information, returned by sp_lock system stored procedure needs in some clarification, because it's difficult to understand database name, object name and index name by their ID numbers. Check the link below if you need to get user name, host name, database name, index name object name and object owner instead of their ID numbers: <a href="http://www.mssqlcity.com/Articles/Adm/LockView.htm">Detailed locking view: sp_lock2</a> <h2><a name="part_7"></a>Literature</h2>1. SQL Server Books Online 2. <a href="http://www.mssqlcity.com/Articles/General/TIL.htm">Transaction Isolation Level</a> 3. <a href="http://www.mssqlcity.com/Articles/SQL65/SQL65Locks.htm">Locking in SQL Server 6.5</a> 4. <a href="http://www.mssqlcity.com/Articles/Adm/LockView.htm">Detailed locking view: sp_lock2</a> 5. <a href="http://support.microsoft.com/support/kb/articles/Q169/9/60.ASP" target="_blank">INF: Analyzing and Avoiding Deadlocks in SQL Server</a></li> <hr> Article By: <h1>Alexander Chigrik <a href="mailto:chigrik@mssqlcity.com">chigrik@mssqlcity.com</a></h1> <h1><a href="http://www.mssqlcity.com/Articles/Adm/SQL70Locks.htm">http://www.mssqlcity.com/Articles/Adm/SQL70Locks.htm</a></h1> <h1> <hr> </h1></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></li></div> <div class="blogger-post-footer"></div> </article> <article> <h1>DIV Rounded Corners</h1> 2008-07-17T20:52:00.001+06:00 <div dir="ltr"><h3>CSS</h3> <pre><style type="text/css"> .spiffy{display:block} .spiffy *{ display:block; height:1px; overflow:hidden; font-size:.01em; background:#B3A400} .spiffy1{ margin-left:3px; margin-right:3px; padding-left:1px; padding-right:1px; border-left:1px solid #ded791; border-right:1px solid #ded791; background:#c6ba3f} .spiffy2{ margin-left:1px; margin-right:1px; padding-right:1px; padding-left:1px; border-left:1px solid #f7f5e5; border-right:1px solid #f7f5e5; background:#c1b530} .spiffy3{ margin-left:1px; margin-right:1px; border-left:1px solid #c1b530; border-right:1px solid #c1b530;} .spiffy4{ border-left:1px solid #ded791; border-right:1px solid #ded791} .spiffy5{ border-left:1px solid #c6ba3f; border-right:1px solid #c6ba3f} .spiffyfg{ background:#B3A400} </style> </pre> <h3>HTML</h3> <pre><div> <div class="spiffyfg">  </div> </div> </pre></div> <div class="blogger-post-footer"></div> </article> <article> <h1>Convert DOC to PDF Files (Convert Word To PDF Files) using Command in ASP.Net</h1> 2008-07-16T18:53:00.002+06:00 <div dir="ltr">I used <a href="http://www.softinterface.com/Convert-Doc/Features/Convert-DOC-to-PDF.htm">http://www.softinterface.com/Convert-Doc/Features/Convert-DOC-to-PDF.htm</a> and then Executed the command from Asp.Net to conver the MS Word files to PDF. Code // write the command string exec = TextBox1.Text; // Create the ProcessInfo object System.Diagnostics.ProcessStartInfo psi = new System.Diagnostics.ProcessStartInfo("cmd.exe"); psi.UseShellExecute = false; psi.RedirectStandardOutput = true; psi.RedirectStandardInput = true; psi.RedirectStandardError = true; // The path where Application is installed psi.WorkingDirectory = "C:\\Program Files\\Softinterface, Inc\\Convert Doc"; // Start the process System.Diagnostics.Process proc = System.Diagnostics.Process.Start(psi); // Open the batch file for reading //System.IO.StreamReader strm = System.IO.File.OpenText(strFilePath); System.IO.StreamReader strm = proc.StandardError; // Attach the output for reading System.IO.StreamReader sOut = proc.StandardOutput; // Attach the in for writing System.IO.StreamWriter sIn = proc.StandardInput; sIn.WriteLine(exec); strm.Close(); sIn.WriteLine("EXIT"); // Close the process proc.Close(); // Close the io Streams; sIn.Close(); sOut.Close(); ASPX File <div> <asp:TextBox ID="TextBox1" runat="server"></asp:TextBox> <asp:Button ID="Button1" runat="server" OnClick="Button1_Click" Text="Button" /></div> Commands ( Ref: <a href="http://www.softinterface.com/">http://www.softinterface.com/</a> ) <h2 style="font-family: arial,sans-serif;">Converting a Single File</h2> To convert a single file, say D:\MyFolder\Doc1.DOC to C:\Results Folder\Doc1.PDF use the following syntax: ConvertDoc /S "D:\MyFolder\Doc1.DOC" /T "C:\Results Folder\Doc1.PDF" /F9 /C12 /M2 /V <ul style="font-family: arial,sans-serif;"><li> The /S and /T switches above specify Source (input) and Target (output) path respectively and are both required when converting a single file. It is always a good idea to use double quotes around the path especially if there are space characters within the path. </li><li> The /M2 switch tells 'Convert Doc' to use the 'Convert Doc' method (it is one of 3 different possible Conversion Methods). </li><li> /F9 is the original (input) file type, which in this case is a word DOC file. Looking up the file types within the File Type Constants Specification for the 'Convert Doc' method will show that the numeric value of 9 corresponds to a DOC file. </li><li> /C12 is the target (output) file type, which in this case is a PDF file. Looking up the file types within the File Type Constants Specification for the 'Convert Doc' method will show that the numeric value of 12 corresponds to a PDF file. </li><li> Finally, the /V (for Verbose) switch is used to give instant feedback by having the program report the status of the conversion with a message box. You can remove this once you have perfected your command line specification. You can also (or instead of /V) create a Log file that will contain the results of the conversion by using the /L switch. </li><li> HINT: You may use the /W switch to specify a File Open password. The Example below makes the word Apples the password to open the newly created PDF file: ConvertDoc.EXE /S "c:\input files\tryme.doc" /T "c:\input files\tryme.pdf" /F9 /C12 /M2 /V /WApples </li></ul> <div style="border-style: solid none; border-color: windowtext -moz-use-text-color; border-width: 1pt medium; padding: 1pt 0in; margin-left: 5.75pt; margin-right: 0in; font-family: arial,sans-serif;"> Note: It is highly encouraged that you use the Verbose (/V) switch initially to see what the status of your conversion is and to help you perfect your command line. When in verbose mode the program will tell you what went wrong or right with your command line using message boxes.</div></div><div class="blogger-post-footer"></div> </article> <article> <h1>Numeric TextBox (ASP.Net / HTML Input)</h1> 2008-07-16T12:45:00.001+06:00 <div dir="ltr"> <div> </div> <div><</div>script language="javascript"></div> function KeyCheck(e) { var KeyID = (window.event) ? event.keyCode : e.which; if((KeyID >= 65 && KeyID <= 90) || (KeyID >= 97 && KeyID <= 122) || (KeyID >= 33 && KeyID <= 47) || (KeyID >= 58 && KeyID <= 64) || (KeyID >= 91 && KeyID <= 96) || (KeyID >= 123 && KeyID <= 126)) { return false; } return true; } <div></</div>script> <div> </div> <div> </div> <div> </div> <div>HTML Text Box</div> <div> </div> <div> <input type="text" ID="txt_TextBox" onkeypress="return KeyCheck(event);" runat="server" style="width: 28px" /> ASP.Net TextBox <asp:TextBox ID="txt_ASPTextBox" runat="server" ></asp:TextBox> Code Behind for Asp.Net TextBox txt_ASPTextBox.Attributes.Add("onkeypress","return KeyCheck(event);");</div><div class="blogger-post-footer"></div> </article> <article> <h1>Insert / Delete / Update Row in DataGrid at Runtime from Other GridView</h1> 2008-07-16T12:28:00.001+06:00 <div dir="ltr"> <div>Grid 1</div> <div><asp:GridView ID="gv_Source" runat="server" AutoGenerateColumns="False" OnRowCommand="gv_Source_RowCommand" Width="100%"></div> <Columns> <asp:BoundField DataField="TemplateName" HeaderText="Template Name" /> <asp:TemplateField> <ItemTemplate> <input type="text" ID="txt_TextBox" runat="server" style="width: 28px" /> <asp:Label ID="lbl_Required" runat="server" Font-Bold="True" ForeColor="Red" Text="*" Visible="False"></asp:Label> </ItemTemplate> </asp:TemplateField> <asp:TemplateField> <ItemTemplate> <asp:LinkButton ID="lnkbt_SelectTemplates" runat="server" Text="Select" CommandArgument='<%# ((GridViewRow)Container).RowIndex %>' ValidationGroup="SelectGroup"></asp:LinkButton> </ItemTemplate> </asp:TemplateField> </Columns> </asp:GridView> <div> </div> <div>Grid2</div> <div> </div> <div><asp:GridView ID="gv_Selected" runat="server" AutoGenerateColumns="False" HorizontalAlign="Left" Width="100%" OnRowCommand="gv_Selected_RowCommand"></div> <Columns> <asp:BoundField DataField="Column1" HeaderText="Column Name" /> <asp:BoundField DataField="Column2" HeaderText="Column2 Nam" /> <asp:TemplateField> <ItemTemplate> <asp:LinkButton ID="lnkbt_Delete" runat="server" Text="Remove" CommandArgument='<%# ((GridViewRow)Container).RowIndex %>' CausesValidation="False" OnClientClick="return confirm('Are you sure?');"></asp:LinkButton> </ItemTemplate></asp:TemplateField> </Columns> </asp:GridView> <div></div> </div> <div>CODE BEHIND</div> <div> </div> <div>Grid 1 RowCommand (Add to Grid 2)</div> <div> </div> <div> protected void gv_Source_RowCommand(object sender, GridViewCommandEventArgs e) { GridViewRow gv_row = gv_Source.Rows[int.Parse(e.CommandArgument.ToString())]; DataTable dt = new DataTable("Selected"); dt.Columns.Add("Column1"); dt.Columns.Add("Column2"); dt.AcceptChanges(); DataRow dr = null; //Check if the grid has already some rows if (gv_Selected.Rows.Count > 0) { foreach (GridViewRow gr in gv_Selected.Rows) { dr = dt.NewRow(); dr[0] = gr.Cells[0].Text; dr[1] = gr.Cells[1].Text; dt.Rows.Add(dr); dt.AcceptChanges(); } } if (((HtmlInputText)gv_row.FindControl("txt_TextBox")).Value.Length != 0) { dr = dt.NewRow(); dr[0] = gv_row.Cells[0].Text; dr[1] = ((HtmlInputText)gv_row.FindControl("txt_TextBox")).Value; dt.Rows.Add(dr); dt.AcceptChanges(); Label lbl_temp = (Label)gv_row.FindControl("lbl_Required"); lbl_temp.Visible = false; } else { Label lbl_temp = (Label)gv_row.FindControl("lbl_Required"); lbl_temp.Visible = true; } gv_Selected.DataSource = dt; gv_Selected.DataBind(); } Grid 2 RowCommand (Delete Row) protected void gv_Selected_RowCommand(object sender, GridViewCommandEventArgs e) { DataTable dt = new DataTable("Selected"); dt.Columns.Add("Column1"); dt.Columns.Add("Column2"); dt.AcceptChanges(); DataRow dr = null; if (gv_Selected.Rows.Count > 0) { for (int i = 0; i < gv_Selected.Rows.Count;i++) { GridViewRow gr = gv_SelectedTemplates.Rows[i]; if (i.ToString() != e.CommandArgument.ToString()) { dr = dt.NewRow(); dr[0] = gr.Cells[0].Text; dr[1] = gr.Cells[1].Text; dt.Rows.Add(dr); dt.AcceptChanges(); } } } gv_SelectedTemplates.DataSource = dt; gv_SelectedTemplates.DataBind(); }</div><div class="blogger-post-footer"></div> </article> <article> <h1>GridView (Multiple check Boxes Selection / Get Checked Check Boxes)</h1> 2008-07-14T19:22:00.001+06:00 <div>Grid View</div> <div> </div> <div> <asp:GridView ID="GridView1" runat="server" AutoGenerateColumns="False"> <Columns> <asp:BoundField DataField="Column1" HeaderText="Column" /> <asp:TemplateField HeaderText="TextBox"> <ItemTemplate> <asp:TextBox ID="txt_TextBox" runat="server"></asp:TextBox> </ItemTemplate> </asp:TemplateField> <asp:TemplateField HeaderText="Select"> <HeaderTemplate> Select All<input id="chkAll" onclick="javascript:SelectAllCheckboxes(this);" runat="server" type="checkbox" /> </HeaderTemplate> <ItemTemplate> <asp:CheckBox ID="chk_Selected" runat="server" /> </ItemTemplate> </asp:TemplateField> </Columns> </asp:GridView> JavaScript <script language="javascript"> function SelectAllCheckboxes(spanChk){ var oItem = spanChk.children; var theBox= (spanChk.type=="checkbox") ? spanChk : spanChk.children.item[0]; xState=theBox.checked; elm=theBox.form.elements; for(i=0;i<elm.length;i++) if(elm[i].type=="checkbox" && elm[i].id!=theBox.id) { if(elm[i].checked!=xState) elm[i].click(); } } </script> Code Behind string str_Result = ""; for (int i = 0; i < gv_SearchedDocuments.Rows.Count; i++) { GridViewRow gv_row = GridView1.Rows[i]; bool isChecked = ((CheckBox)gv_row.FindControl("chk_Selected")).Checked; if (isChecked) { str_Result += GridView1.Rows[i].Cells[0].Text; TextBox txt_Temp = ((TextBox)gv_row.FindControl("txt_ViewOrder")); str_Result += txt_Temp.Text+"<,br/>"; } } lbl_Message.Text = str_Result ;</div><div class="blogger-post-footer"></div> </article> <article> <h1>Exception Handling (Exception logging)</h1> 2008-07-08T12:33:00.001+06:00 using System; using System.Collections.Generic; using System.Text; using System.Diagnostics; namespace ExceptionLogger { public class ExceptionLogger { public static void WritetoCustomEventLog(Exception paramEntry, EventLogEntryType paramType) { string str_Source = "ERROR LOG"; //Source Name string str_LogType = "My Test LOG"; //Your log Name if it does not exists then first it will create and then add entry string str_Machine = "."; //machine name if (!System.Diagnostics.EventLog.SourceExists(str_Source, str_Machine)) { System.Diagnostics.EventLog.CreateEventSource(str_Source, str_LogType, str_Machine); } string ExceptionDetail = ""; ExceptionDetail = "Message : " + paramEntry.Message + "\n"; ExceptionDetail += "Stack Trace : " + paramEntry.StackTrace + "\n"; ExceptionDetail += "Source : " + paramEntry.Source + "\n"; ExceptionDetail += "Target Site:" + paramEntry.TargetSite + "\n"; ExceptionDetail += "Inner Exception:" + paramEntry.InnerException + "\n"; ExceptionDetail += "Help Link:" + paramEntry.HelpLink + "\n"; EventLog CustomLog = new EventLog(str_LogType, str_Machine, str_Source); CustomLog.WriteEntry(ExceptionDetail, paramType); } } }<div class="blogger-post-footer"></div> </article> <article> <h1>Access Master Pages Properties</h1> 2008-07-07T21:43:00.001+06:00 There are different techniques to access MasterPages properties Master Page Code file: public partial class MyMasterPageClass: System.Web.UI.MasterPage { public string ErrorMessage { set { lblErrorPane.Text = value; panelErrorMessage.Visible = true; } } } Content Page 1) ((MyMasterPageClass)Master).ErrorMessage = " Test Error Message"; After adding this <%@ MasterType VirtualPath="~/MasterPage.master" %> under Page Directive you can access the property of master page directly. 2) this.Master.ErrorMessage = "Test Message"; <div class="blogger-post-footer"></div> </article> <article> <h1>Unique String Generator</h1> 2008-07-04T00:38:00.001+06:00 using System; using System.Security.Cryptography; public class UniqueStringGenerator { private static int DEFAULT_MIN_STRING_LENGTH = 20; private static int DEFAULT_MAX_STRING_LENGTH = 20; private static string STRING_CHARS_LCASE = "abcdefgijkmnopqrstwxyz"; private static string STRING_CHARS_UCASE = "ABCDEFGHJKLMNPQRSTWXYZ"; private static string STRING_CHARS_NUMERIC = "23456789"; private static string STRING_CHARS_SPECIAL = "*$-+?_&=!%{}/"; public static string Generate() { return Generate(DEFAULT_MIN_STRING_LENGTH, DEFAULT_MAX_STRING_LENGTH); } public static string Generate(int length) { return Generate(length, length); } public static string GenerateWithSpecialCharacters() { return GenerateWithSpecialCharacters(DEFAULT_MIN_STRING_LENGTH, DEFAULT_MAX_STRING_LENGTH); } public static string GenerateWithSpecialCharacters(int length) { return GenerateWithSpecialCharacters(length, length); } public static string GenerateWithSpecialCharacters(int minLength, int maxLength) { if (minLength <= 0 || maxLength <= 0 || minLength > maxLength) return null; char[][] charGroups = new char[][] { STRING_CHARS_LCASE.ToCharArray(), STRING_CHARS_UCASE.ToCharArray(), STRING_CHARS_NUMERIC.ToCharArray(), STRING_CHARS_SPECIAL.ToCharArray() }; int[] charsLeftInGroup = new int[charGroups.Length]; for (int i = 0; i < charsLeftInGroup.Length; i++) charsLeftInGroup[i] = charGroups[i].Length; int[] leftGroupsOrder = new int[charGroups.Length]; for (int i = 0; i < leftGroupsOrder.Length; i++) leftGroupsOrder[i] = i; byte[] randomBytes = new byte[4]; RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider(); rng.GetBytes(randomBytes); int seed = (randomBytes[0] & 0x7f) << 24 | randomBytes[1] << 16 | randomBytes[2] << 8 | randomBytes[3]; Random random = new Random(seed); char[] UniqueString = null; if (minLength < maxLength) UniqueString = new char[random.Next(minLength, maxLength + 1)]; else UniqueString = new char[minLength]; int nextCharIdx; int nextGroupIdx; int nextLeftGroupsOrderIdx; int lastCharIdx; int lastLeftGroupsOrderIdx = leftGroupsOrder.Length - 1; for (int i = 0; i < UniqueString.Length; i++) { if (lastLeftGroupsOrderIdx == 0) nextLeftGroupsOrderIdx = 0; else nextLeftGroupsOrderIdx = random.Next(0, lastLeftGroupsOrderIdx); nextGroupIdx = leftGroupsOrder[nextLeftGroupsOrderIdx]; lastCharIdx = charsLeftInGroup[nextGroupIdx] - 1; if (lastCharIdx == 0) nextCharIdx = 0; else nextCharIdx = random.Next(0, lastCharIdx + 1); UniqueString[i] = charGroups[nextGroupIdx][nextCharIdx]; if (lastCharIdx == 0) charsLeftInGroup[nextGroupIdx] = charGroups[nextGroupIdx].Length; else { if (lastCharIdx != nextCharIdx) { char temp = charGroups[nextGroupIdx][lastCharIdx]; charGroups[nextGroupIdx][lastCharIdx] = charGroups[nextGroupIdx][nextCharIdx]; charGroups[nextGroupIdx][nextCharIdx] = temp; } charsLeftInGroup[nextGroupIdx]--; } if (lastLeftGroupsOrderIdx == 0) lastLeftGroupsOrderIdx = leftGroupsOrder.Length - 1; else { if (lastLeftGroupsOrderIdx != nextLeftGroupsOrderIdx) { int temp = leftGroupsOrder[lastLeftGroupsOrderIdx]; leftGroupsOrder[lastLeftGroupsOrderIdx] = leftGroupsOrder[nextLeftGroupsOrderIdx]; leftGroupsOrder[nextLeftGroupsOrderIdx] = temp; } lastLeftGroupsOrderIdx--; } } return new string(UniqueString); } public static string Generate(int minLength, int maxLength) { if (minLength <= 0 || maxLength <= 0 || minLength > maxLength) return null; char[][] charGroups = new char[][] { STRING_CHARS_LCASE.ToCharArray(), STRING_CHARS_UCASE.ToCharArray(), STRING_CHARS_NUMERIC.ToCharArray(), }; int[] charsLeftInGroup = new int[charGroups.Length]; for (int i = 0; i < charsLeftInGroup.Length; i++) charsLeftInGroup[i] = charGroups[i].Length; int[] leftGroupsOrder = new int[charGroups.Length]; for (int i = 0; i < leftGroupsOrder.Length; i++) leftGroupsOrder[i] = i; byte[] randomBytes = new byte[4]; RNGCryptoServiceProvider rng = new RNGCryptoServiceProvider(); rng.GetBytes(randomBytes); int seed = (randomBytes[0] & 0x7f) << 24 | randomBytes[1] << 16 | randomBytes[2] << 8 | randomBytes[3]; Random random = new Random(seed); char[] UniqueString = null; if (minLength < maxLength) UniqueString = new char[random.Next(minLength, maxLength + 1)]; else UniqueString = new char[minLength]; int nextCharIdx; int nextGroupIdx; int nextLeftGroupsOrderIdx; int lastCharIdx; int lastLeftGroupsOrderIdx = leftGroupsOrder.Length - 1; for (int i = 0; i < UniqueString.Length; i++) { if (lastLeftGroupsOrderIdx == 0) nextLeftGroupsOrderIdx = 0; else nextLeftGroupsOrderIdx = random.Next(0, lastLeftGroupsOrderIdx); nextGroupIdx = leftGroupsOrder[nextLeftGroupsOrderIdx]; lastCharIdx = charsLeftInGroup[nextGroupIdx] - 1; if (lastCharIdx == 0) nextCharIdx = 0; else nextCharIdx = random.Next(0, lastCharIdx + 1); UniqueString[i] = charGroups[nextGroupIdx][nextCharIdx]; if (lastCharIdx == 0) charsLeftInGroup[nextGroupIdx] = charGroups[nextGroupIdx].Length; else { if (lastCharIdx != nextCharIdx) { char temp = charGroups[nextGroupIdx][lastCharIdx]; charGroups[nextGroupIdx][lastCharIdx] = charGroups[nextGroupIdx][nextCharIdx]; charGroups[nextGroupIdx][nextCharIdx] = temp; } charsLeftInGroup[nextGroupIdx]--; } if (lastLeftGroupsOrderIdx == 0) lastLeftGroupsOrderIdx = leftGroupsOrder.Length - 1; else { if (lastLeftGroupsOrderIdx != nextLeftGroupsOrderIdx) { int temp = leftGroupsOrder[lastLeftGroupsOrderIdx]; leftGroupsOrder[lastLeftGroupsOrderIdx] = leftGroupsOrder[nextLeftGroupsOrderIdx]; leftGroupsOrder[nextLeftGroupsOrderIdx] = temp; } lastLeftGroupsOrderIdx--; } } return new string(UniqueString); } } There are two functions * Generate o This function returns the string with no special characters * GenerateWithSpecialCharacters o This function returns the string with special characters included * Example Code o string temp = UniqueStringGenerator.GenerateWithSpecialCharacters(20,20); o string temp = UniqueStringGenerator.GenerateWithSpecialCharacters(20); o string temp = UniqueStringGenerator.GenerateWithSpecialCharacters(); o ....................... o string temp = UniqueStringGenerator.Generate(20,20); o string temp = UniqueStringGenerator.Generate(20); o string temp = UniqueStringGenerator.Generate();<div class="blogger-post-footer"></div> </article> <article> <h1>Spider</h1> 2008-04-02T21:02:00.000+05:00 Download Spider <div class="blogger-post-footer"></div> </article> <article> <h1>ASP.Net Localization</h1> 2008-03-24T19:08:00.001+05:00 To use resources first create Resource files for a page, to generate resources to go Tools - > Generate Local Resources. A resource file for that page will be generated in App_LocalResources Folder, for example if i have generated a file against Default.aspx then a resource file will be generated in App_LocalResources with the name Default.aspx.resx, this will be the default resource file having mapping of all the controls texts in the resource. Now Create other resource file with the value in other language against same keys. eg i want to use arabic and i created a file with the name Default.aspx.ar-EG.resx. now to use this resource file i just have to override InitalizeCulture method and have to set the culture value the code is given below. To set the culture and UI culture for an <a href="http://ASP.NET">ASP.NET</a> Web page programmatically 1. Override the InitializeCulture method for the page. 2. In the overridden method, determine which language and culture to set the page to. 3. Set the UI culture and culture in one of the following ways: <ul><li>Set the Culture and UICulture properties of the page to the language and culture string (for example, en-US). These properties are internal to the page, and can only be used in a page.</li> <li>Set the CurrentUICulture and CurrentCulture properties of the current thread to the UI culture and culture, respectively. The CurrentUICulture property takes a language and culture information string. To set the CurrentCulture property, you create an instance of the CultureInfo class and call its CreateSpecificCulture method.</li> </ul>Following is the code to override InitializeCulture. protected override void InitializeCulture() { if (Session["UICulture"] != null) { if (Session["UICulture"].ToString() == "ar-EG") { Thread.CurrentThread.CurrentUICulture = new CultureInfo(Session["UICulture"].ToString()); Thread.CurrentThread.CurrentCulture = CultureInfo.CreateSpecificCulture(Session["UICulture"].ToString()); } } } Add Following code to your Event ( Language select from dropdownlist or button ) protected void Button2_Click(object sender, EventArgs e) { Session["UICulture"] = "ar-EG"; SelectedLanguage.Value = "ar-EG"; //Force re-initialization of the page to fire InitializeCulture() Response.Redirect(Request.Url.LocalPath); } <div class="blogger-post-footer"></div> </article> <article> <h1>YouTube Blocked in Pakistan. Why?</h1> 2008-02-22T17:17:00.000+05:00 It seems that YouTube is down in Pakistan. Possibly it is blocked. Blocking websites is a rather bad habit that the government authorities in Pakistan have gotten into (here and here). This is something that we have written about before. It is silly habit, because technologically it does not work. But, much more importantly, it is a disturbing habit because it points towards a desire by those in authority to “control” the passions of the public. Democracy demands the very opposite - that the people control teh passions of those in power. But the real question is, that if YouTube is blocked, then why?<div class="blogger-post-footer"></div> </article> </main></body></html>

Rizwan's BLOG

Trouble In the House of Google