Pligg long running queries

Submitted by tomo on September 7, 2010 - 2:20am

 

 

For awhile I've been plagued with these long running queries on my Pligg site that were running up CPU:
 
SELECT DISTINCT pv_user_id, pv_type, pv_page_id FROM mypligg_pageviews GROUP BY pv_type, pv_page_id, pv
 
And so I was running the mkill Perl script to kill any queries taking longer than a minute to complete.
But recently even that stopped working, leaving queries in the Killed state but still running up CPU.  They were all stuck in "converting HEAP to MyISAM".  Did some diagnostics:
 
mysql> SHOW GLOBAL STATUS LIKE 'Created_tmp_tables';
+--------------------+-------+
| Variable_name      | Value |
+--------------------+-------+
| Created_tmp_tables | 196   |
+--------------------+-------+
1 row in set (0.01 sec)
 
mysql> SHOW GLOBAL STATUS LIKE 'Created_tmp_disk_tables';
+-------------------------+-------+
| Variable_name           | Value |
+-------------------------+-------+
| Created_tmp_disk_tables | 159   |
+-------------------------+-------+
1 row in set (0.01 sec)
 
mysql> show variables like 'tmp_table_size';
+----------------+----------+
| Variable_name  | Value    |
+----------------+----------+
| tmp_table_size | 33554432 |
+----------------+----------+
1 row in set (0.01 sec)
 
 
So it seemed there wasn't enough memory to perform the conversion without going to disk which is not what we want.  So I set tmp_table_size and max_heap_table_size to larger values, max_heap_table_size too because MySQL uses the lesser of the two.
 
mysql> set tmp_table_size=100000000;
Query OK, 0 rows affected (0.00 sec)
 
mysql> set  max_heap_table_size=100000000;
Query OK, 0 rows affected (0.01 sec)
 

And now we're golden, CPU back down to normal levels and queries finishing.

Read the rest of this article...
November 13, 2010 - 12:27am
#2
tomo

To see what's going on first, run:

show processlist;

and look for the "converting HEAP to MyISAM" message.

Top
© 2010-2014 Saigonist.

About the author

Asian-American expat living in Vietnam over 3 years. I am involved in tech startups and do Drupal consulting. I choose to live in Saigon and Vietnam and I want to see the city and its people reach their true potential.

Where is Saigon?

It's not wrong to wonder where in the world Saigon is.

Saigon is the name of the city now officially known as Ho Chi Minh City. But most people who call Saigon home call their home Saigon and it's mostly tourists who I hear refer to it as Ho Chi Minh City (or Ho Chi Minh Ville for the French).

After the Vietnam/American War ended in 1975, many place names ceased to exist such as Saigon, South Vietnam, and plenty of street names.

Tu Do is now Dong Khoi, Thong Nhat is Le Duan. And Tan Son Nhut Airport is now Tan Son Nhat International Airport. Rue Paul Blanchy has been Hai Ba Trung since the 1950s.

Those who haven't seen Vietnam for decades will be confused not just by the geographical changes but the incredible amount of development and modernization that has since taken place.

Latest Technical Posts

I also post a lot of technical material unrelated to Vietnam that doesn't hit the home page. Starting from November 2015 I am only posting technical articles on my new sub-blog: TECH.saigonist.com

  • Definition of Startup: What isn't a startup? : What is a startup? We're in the middle of a tech boom and it's become fashionable to either work for a startup or to be a startup. But what is a startup? How do you know that company offering you a...
  • My Startup Experiences : I've been involved in start ups for over a decade now. During that time I've worked for companies in varying stage of development. Not all of them were startups. But how can you tell if it is or not...
  • Code Retreat 2012 - Ho Chi Minh City : $(document).ready(function(){ $('.geshifilter').hover(function(){console.log(this);console.log($(this));$(this).css('position', 'relative').css('width', '800px').css('z-index', 1000);}, function...
  • Internet Censorship in Southeast Asian Countries (Myanmar, China, Malaysia, Thailand, Cambodia, Vietnam) : Vietnam has been labeled an "enemy of the Internet" by Reporters Without Borders. There have been a lot of cases of bloggers being targeted, harrassed, and arrested. But RSF (Reporters Sans...
  • Using 3G Internet in Cambodia : On my most recent trip to Cambodia I bought a Cambodian SIM card so that I could both make calls and use the Internet on my smartphone. At the Moc Bai/Bavet border are a number of Khmer people (who...
  • MailChimp vs poMMo, Mailman, Phplist - Self-Host or Outsource Your Mailing List? : Use your own mail server or outsource it? Let's say you have a rather large mailing list of a few thousand email addresses and, well, sending it out with GMail is no longer cutting it. What are your...
  • Internet in Vietnam - Good, Bad, Ugly : If you ask Vietnamese officials, you'll hear such hyperbole as "the Internet service development in Vietnam is in no way inferior to developed countries in the world" (Mai Liem Truc, former Deputy...
  • poMMo proBLEMs : poMMo is free open source mailing list management software. It's written in PHP and pretty basic but does give a decent management interface. However, it's no longer maintained. There is a GitHub...
  • Preventing Repetitive Stress Injuries for Programmers and Writers : My situation I'm a career programmer who has suffered from pain in my hands and arms for several years. Like carpal tunnel syndrome, this pain is from damage to tissue (including tendons) caused by...
  • Calling the US for free from foreign countries : Calling US numbers from the US is pretty trivial and nearly free. But calling from another country means either: - paying for long distance from your telephone company - using a phone card to call...

Recent activity in the expat forum...