Архивы django

Django New Relic uWSGI

Устанавливаем New Relic

pip install newrelic

1

pip install newrelic
Лицензионный ключ берем из личного кабинета, генерируем конфигурацию

newrelic-admin generate-config LICENSE-KEY newrelic.ini

1

newrelic-admin generate-config LICENSE-KEY newrelic.ini

В конфигурации правим название приложения

# The appplication name. Set this to be the name of your
# application as you would like it to show up in New Relic UI.
# The UI will then auto-map instances of your application into a
# entry on your home dashboard page.
app_name = app.name

# The appplication name. Set this to be the name of your

# application as you would like it to show up in New Relic UI.

# The UI will then auto-map instances of your application into a

# entry on your home dashboard page.

app_name = app.name

Модифицируем project/wsgi.py

import os
import newrelic.agent
newrelic.agent.initialize('/path/to/newrelic.ini')
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
application = newrelic.agent.wsgi_application()(application)

import os

import newrelic.agent

newrelic.agent.initialize('/path/to/newrelic.ini')

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")

from django.core.wsgi import get_wsgi_application

application = get_wsgi_application()

application = newrelic.agent.wsgi_application()(application)

Правим конфигурацию uwsgi.ini

#module = django.core.handlers.wsgi:WSGIHandler() module = project.wsgi eval = import newrelic.agent, wsgi; application = newrelic.agent.wsgi_application()(wsgi.application)

1
2
3

#module = django.core.handlers.wsgi:WSGIHandler()
module = project.wsgi
eval = import newrelic.agent, wsgi; application = newrelic.agent.wsgi_application()(wsgi.application)

sql_joined_field sphinx Django m2m

Необходимо включить результаты поиска по полю name, связанной m2m модели Category.

Модели:

class Category(models.Model):
name = models.CharField(_(u'Название категории'), max_length=60, unique=True)
...
class Product(models.Model):
...
category = models.ManyToManyField(Category, verbose_name=u'Категория', related_name='products')
...

class Category(models.Model):

name = models.CharField(_(u'Название категории'), max_length=60, unique=True)

...

class Product(models.Model):

...

category = models.ManyToManyField(Category, verbose_name=u'Категория', related_name='products')

...

Конфиг сфинкса:

sql_attr_multi = uint category from query; SELECT product_id, category_id FROM core_product_category
sql_joined_field = category from query; SELECT core_product_category.product_id, name \ 
FROM core_category join core_product_category on core_category.id=core_product_category.category_id order by core_product_category.product_id ASC

sql_attr_multi = uint category from query; SELECT product_id, category_id FROM core_product_category

sql_joined_field = category from query; SELECT core_product_category.product_id, name \

FROM core_category join core_product_category on core_category.id=core_product_category.category_id order by core_product_category.product_id ASC

ascii codec can’t encode characters in position «» Django Supervisor

В конфиг supervisor добавляем

environment=LANG="ru_RU.utf8", LC_ALL="ru_RU.UTF-8", LC_LANG="ru_RU.UTF-8"

1	environment=LANG="ru_RU.utf8", LC_ALL="ru_RU.UTF-8", LC_LANG="ru_RU.UTF-8"

При обновлении MySQL миниатюры изображений, отрисованные с помощью Pillow и easy_thumbnails для Django, могут перестать работать, рецепт такой же, добавить локаль в конфиг supervisor и перезапустить его.

Django Ajax

Нашел чудесную библиотеку django-ajax, берет на себя всю рутину, связанную с ajax во вьюшках Django.
Вьюшка теперь может выглядеть так:

from django_ajax.decorators import ajax
from models import Product
from cart.cart import Cart
@ajax
def ajax_add_to_cart(request):
if 'product_id' in request.GET and request.GET['product_id']:
product_id = request.GET['product_id']
product = Product.objects.get(id=product_id)
cart = Cart(request)
cart.add(product, product.price, quantity=1)
items_in_cart = cart.itemCount()
return {'items_in_cart': items_in_cart}

from django_ajax.decorators import ajax

from models import Product

from cart.cart import Cart

@ajax

def ajax_add_to_cart(request):

if 'product_id' in request.GET and request.GET['product_id']:

product_id = request.GET['product_id']

product = Product.objects.get(id=product_id)

cart = Cart(request)

cart.add(product, product.price, quantity=1)

items_in_cart = cart.itemCount()

return {'items_in_cart': items_in_cart}

Ответом будет JSON такого вида

{"status": 200, "statusText": "OK", "content": {"items_in_cart": 5}}

1	{"status": 200, "statusText": "OK", "content": {"items_in_cart": 5}}

Шаблон может выглядеть так:

<a class="in-cart" href="{% url 'add_to_cart' item.id 1 %}">в корзину</a>
<script type="text/javascript">
$(function() {
var lnk = $("a.in-cart");
$(lnk).click(function() {
$.get($(lnk).attr('href')).done(function( json ) {
$("#items_in_cart").text(json.content.items_in_cart)
});
return false
});
});
</script>

<a class="in-cart" href="{% url 'add_to_cart' item.id 1 %}">в корзину</a>

$(function() {

var lnk = $("a.in-cart");

$(lnk).click(function() {

$.get($(lnk).attr('href')).done(function( json ) {

$("#items_in_cart").text(json.content.items_in_cart)

});

return false

});

</script>

django-simple-history трекинг истории изменения объекта

В джанго есть встроенный трекер истории, но он не позволяет откатиться на предыдущее состояние. Решить эту задачу можно с помощью удобного инструмента django-simple-history.

pip install django-simple-history

1	pip install django-simple-history

В settings.py

INSTALLED_APPS = (...
'simple_history',
...)

INSTALLED_APPS = (...

'simple_history',

...)

В models.py

from simple_history.models import HistoricalRecords
class Product(models.Model):
...   
history = HistoricalRecords()

from simple_history.models import HistoricalRecords

class Product(models.Model):

...

history = HistoricalRecords()

Выполняем миграцию

./manage.py schemamigration <app_name> --auto
./manage.py migrate <app_name>

1 2	./manage.py schemamigration <app_name> --auto ./manage.py migrate <app_name>

Далее в admin.py

from simple_history.admin import SimpleHistoryAdmin
class ProductAdmin(SimpleHistoryAdmin, admin.ModelAdmin):
...
admin.site.register(Product, ProductAdmin)

from simple_history.admin import SimpleHistoryAdmin

class ProductAdmin(SimpleHistoryAdmin, admin.ModelAdmin):

...

admin.site.register(Product, ProductAdmin)

Теперь при нажатии на кнопку «История» объекта можно откатиться на выбранную ревизию.

Memcached/Johnny Cache Django 1.5 Debian

Ничего нового, просто чтобы не гуглить в очередной раз.

aptitude install memcached

1	aptitude install memcached

pip install python-memcached

1	pip install python-memcached

pip install johnny-cache

1	pip install johnny-cache

Конфиг memcached находится в /etc/memcached.conf, оставляю все значения по умолчанию.

В settings.py

INSTALLED_APPS = (
# ...
'johnny',
)

INSTALLED_APPS = (

# ...

'johnny',

)

MIDDLEWARE_CLASSES = (
'johnny.middleware.LocalStoreClearMiddleware',
'johnny.middleware.QueryCacheMiddleware',
# ...
)

MIDDLEWARE_CLASSES = (

'johnny.middleware.LocalStoreClearMiddleware',

'johnny.middleware.QueryCacheMiddleware',

# ...

)

CACHES = {
'default' : dict(
BACKEND = 'johnny.backends.memcached.MemcachedCache',
LOCATION = ['127.0.0.1:11211'],
JOHNNY_CACHE = True,
)
}
JOHNNY_MIDDLEWARE_KEY_PREFIX='jc_myproj'

CACHES = {

'default' : dict(

BACKEND = 'johnny.backends.memcached.MemcachedCache',

LOCATION = ['127.0.0.1:11211'],

JOHNNY_CACHE = True,

)

}

JOHNNY_MIDDLEWARE_KEY_PREFIX='jc_myproj'

— префикс текущего проекта

Если у вас Django 1.5 и при пустых queryset вываливается ошибка

AttributeError: 'module' object has no attribute 'iter'

1	AttributeError: 'module' object has no attribute 'iter'

ставьте с гитхаба

git clone http://github.com/jmoiron/johnny-cache.git
cd johnny-cache && ./setup.py

git clone http://github.com/jmoiron/johnny-cache.git

cd johnny-cache && ./setup.py

Django + Sphinx + Debian

Ставим отсюда

https://github.com/FactorAG/django-sphinx

1	https://github.com/FactorAG/django-sphinx

В settings.py

INSTALLED_APS= (
...
'djangosphinx',
...
)
# Sphinx 2.0.4/2.0.6/2.0.8
SPHINX_API_VERSION = 0x119

INSTALLED_APS= (

...

'djangosphinx',

...

)

# Sphinx 2.0.4/2.0.6/2.0.8

SPHINX_API_VERSION = 0x119

models.py

from djangosphinx.models import SphinxSearch
class Product(models.Model):
brand = models.ForeignKey(Brand, verbose_name=u'Бренд')
title = models.CharField(_(u'Марка'), max_length=50, blank=True)
caption = models.CharField(_(u'Серия'), max_length=150, blank=True)
composition = models.CharField(_(u'Ингредиенты'), max_length=60, blank=True, null=True)
about = RichTextField(_(u'Описание'), blank=True)
pub_date = models.DateTimeField(default=datetime.now)
pet = models.ManyToManyField(Pet, verbose_name=u'Животное')
updated = models.DateTimeField(auto_now=True)
search = SphinxSearch(weights={'title': 90, 'caption': 80, 'about': 100, 'composition': 70},
mode='SPH_MATCH_ALL', rankmode="SPH_RANK_BM25", sort='SPH_SORT_RELEVANCE',
)

from djangosphinx.models import SphinxSearch

class Product(models.Model):

brand = models.ForeignKey(Brand, verbose_name=u'Бренд')

title = models.CharField(_(u'Марка'), max_length=50, blank=True)

caption = models.CharField(_(u'Серия'), max_length=150, blank=True)

composition = models.CharField(_(u'Ингредиенты'), max_length=60, blank=True, null=True)

about = RichTextField(_(u'Описание'), blank=True)

pub_date = models.DateTimeField(default=datetime.now)

pet = models.ManyToManyField(Pet, verbose_name=u'Животное')

updated = models.DateTimeField(auto_now=True)

search = SphinxSearch(weights={'title': 90, 'caption': 80, 'about': 100, 'composition': 70},

mode='SPH_MATCH_ALL', rankmode="SPH_RANK_BM25", sort='SPH_SORT_RELEVANCE',

)

views.py

@render_to('core/entry_list.html')
def searcher(request):
if 'srch' in request.GET and request.GET['srch']:
txt = request.GET['srch']
entries = Product.search.query(txt)
return {'object_list': entries, }
else:
return root(request)

@render_to('core/entry_list.html')

def searcher(request):

if 'srch' in request.GET and request.GET['srch']:

txt = request.GET['srch']

entries = Product.search.query(txt)

return {'object_list': entries, }

else:

return root(request)

django-sphinx умеет создавать конфигурацию для Sphinx

./manage.py generate_sphinx_config <app_name> > config/sphinx.conf

1	./manage.py generate_sphinx_config <app_name> > config/sphinx.conf

Получившийся файл конфигурации нужно немного дополнить

source core_product
{
type                = mysql
sql_host            =
sql_user            = <DB_user>
sql_pass            = ******
sql_db              = <DB>
sql_port            =
sql_query_pre       = SET NAMES utf8
sql_query_post      =
sql_query           = \
SELECT id, onmain, brand_id, title, caption, gen_caption, composition, about, ext, pub_date, updated\
FROM core_product
sql_query_info      = SELECT * FROM `core_product` WHERE `id` = $id
# ForeignKey's
sql_attr_uint       = brand_id
# DateField's and DateTimeField's
sql_attr_timestamp   = pub_date
sql_attr_timestamp   = updated
}
index core_product
{
source          = core_product
path            = /home/project/var/sphinx
docinfo         = extern
morphology      = stem_enru, Soundex, Metaphone
charset_type    = utf-8
min_word_len    = 2
min_infix_len   = 2
min_prefix_len  = 0
enable_star     = 1
charset_table = 0..9, A..Z->a..z, _, a..z, U+0401->U+0435, U+0451->U+0435, U+410..U+42F->U+430..U+44F, U+430..U+44F
index_exact_words = 1
expand_keywords   = 1
index_sp=1
html_index_attrs = img=alt,title; a=title;
html_strip=1
}
indexer {
mem_limit = 64M
}
searchd {
listen = 9312
listen = 9306:mysql41
log = /home/project/logs/searchd.log
query_log = /project/logs/query.log
read_timeout = 5
client_timeout = 300
max_children = 30
pid_file = /home/project/tmp/searchd.pid
max_matches = 1000
seamless_rotate = 1
preopen_indexes = 1
unlink_old = 1
mva_updates_pool = 1M
max_packet_size = 8M
max_filters = 256
max_filter_values = 4096
max_batch_queries = 32
workers = threads
}

source core_product

{

type = mysql

sql_host =

sql_user = <DB_user>

sql_pass = ******

sql_db = <DB>

sql_port =

sql_query_pre = SET NAMES utf8

sql_query_post =

sql_query = \

SELECT id, onmain, brand_id, title, caption, gen_caption, composition, about, ext, pub_date, updated\

FROM core_product

sql_query_info = SELECT * FROM `core_product` WHERE `id` = $id

# ForeignKey's

sql_attr_uint = brand_id

# DateField's and DateTimeField's

sql_attr_timestamp = pub_date

sql_attr_timestamp = updated

}

index core_product

{

source = core_product

path = /home/project/var/sphinx

docinfo = extern

morphology = stem_enru, Soundex, Metaphone

charset_type = utf-8

min_word_len = 2

min_infix_len = 2

min_prefix_len = 0

enable_star = 1

charset_table = 0..9, A..Z->a..z, _, a..z, U+0401->U+0435, U+0451->U+0435, U+410..U+42F->U+430..U+44F, U+430..U+44F

index_exact_words = 1

expand_keywords = 1

index_sp=1

html_index_attrs = img=alt,title; a=title;

html_strip=1

}

indexer {

mem_limit = 64M

}

searchd {

listen = 9312

listen = 9306:mysql41

log = /home/project/logs/searchd.log

query_log = /project/logs/query.log

read_timeout = 5

client_timeout = 300

max_children = 30

pid_file = /home/project/tmp/searchd.pid

max_matches = 1000

seamless_rotate = 1

preopen_indexes = 1

unlink_old = 1

mva_updates_pool = 1M

max_packet_size = 8M

max_filters = 256

max_filter_values = 4096

max_batch_queries = 32

workers = threads

}

Стваим Sphinx

Качаем исходники (есть готовые пакеты) отсюда

http://sphinxsearch.com/downloads/release/

1

http://sphinxsearch.com/downloads/release/
Распаковываем и конфигурируем

tar xzvf sphinx-2.0.8-release.tar.gz sphinx-2.0.8-release ./configure --with-mysql make

1
2
3
4

tar xzvf sphinx-2.0.8-release.tar.gz
sphinx-2.0.8-release
./configure --with-mysql
make
Осталось собрать пакет и установить его в ситему (никогда не делайте make install, каждый раз, когда вы так делаете, умирает котенок)

aptitude install checkinstall checkinstall

1
2

aptitude install checkinstall
checkinstall

теперь вы можете управлять пакетом с помощью пакетного менеджера.

Sphinx установлен, осталось проиндексировать БД и запустить демона.

indexer --config config/sphinx.conf --all

1	indexer --config config/sphinx.conf --all

Для дальнейшей индексации можно использовать

indexer --config config/sphinx.conf --all --rotate

1	indexer --config config/sphinx.conf --all --rotate

Для запуска

searchd -c config/sphinx.conf

1	searchd -c config/sphinx.conf

Для остановки

searchd --stop -c config/sphinx.conf

1	searchd --stop -c config/sphinx.conf

Для повышения качества поиска можно использовать словарь словоформ

aptitude install myspell-ru
spelldump /usr/share/hunspell/ru_RU.dic /usr/share/hunspell/ru_RU.aff wordforms_ru_RU.txt
cat wordforms_ru_RU.txt | enca -L ru 
iconv -f KOI8-R -t UTF-8 -o wordforms_ru_RU_UTF8.txt wordforms_ru_RU.txt

aptitude install myspell-ru

spelldump /usr/share/hunspell/ru_RU.dic /usr/share/hunspell/ru_RU.aff wordforms_ru_RU.txt

cat wordforms_ru_RU.txt | enca -L ru

iconv -f KOI8-R -t UTF-8 -o wordforms_ru_RU_UTF8.txt wordforms_ru_RU.txt

Затем подключаем его в sphinx.conf в раздел index

wordforms = /path/to/wordforms_ru_RU_UTF8.txt

1	wordforms = /path/to/wordforms_ru_RU_UTF8.txt

Источники
http://sphinxsearch.com/docs/manual-2.0.8.html#supported-system

http://osmanov-dev-notes.blogspot.ru/2011/06/how-to-create-sphinx-wordform.html

http://proft.me/2011/01/22/polnotekstovyj-poisk-v-django/

http://habrahabr.ru/post/136261/

http://habrahabr.ru/post/147745/

http://habrahabr.ru/post/132118/

Пример использования функции reverse() при построении sitemap.xml для Django

Иногда, при построении карты сайта удобно пользоваться reverse() из django.core.urlresolvers

from django.contrib.sitemaps import Sitemap
from django.core.urlresolvers import reverse
from views import root, product_detail, products_list
class RootSitemap(Sitemap):
changefreq = "weekly"
priority = 0.9
def items(self):
return [root]
def location(self, obj):
return reverse(obj)
class ProductDetailSitemap(Sitemap):
changefreq = "weekly"
priority = 0.9
def items(self):
return Products.objects.all()
def location(self, obj):
return reverse(products_list, kwargs={'product_id': obj.pk})
class ProductsListSitemap(Sitemap):
changefreq = "weekly"
priority = 0.5
def items(self):
return Pets.objects.order_by('url_id')
def location(self, obj):
return reverse('products_list', args=[obj.url_id])

from django.contrib.sitemaps import Sitemap

from django.core.urlresolvers import reverse

from views import root, product_detail, products_list

class RootSitemap(Sitemap):

changefreq = "weekly"

priority = 0.9

def items(self):

return [root]

def location(self, obj):

return reverse(obj)

class ProductDetailSitemap(Sitemap):

changefreq = "weekly"

priority = 0.9

def items(self):

return Products.objects.all()

def location(self, obj):

return reverse(products_list, kwargs={'product_id': obj.pk})

class ProductsListSitemap(Sitemap):

changefreq = "weekly"

priority = 0.5

def items(self):

return Pets.objects.order_by('url_id')

def location(self, obj):

return reverse('products_list', args=[obj.url_id])

Установка Django в виртуальное окружение на сервере с CentOS

Подключаем репозиторий EPEL(i686)

wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -ivh epel-release-6-8.noarch.rpm

1 2	wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm rpm -ivh epel-release-6-8.noarch.rpm

Для архитектуры x86_64

wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

1	wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

Устанавливаем python-pip

yum install -y python-pip

1	yum install -y python-pip

Обновляем pip

pip-python install --upgrade pip
Downloading/unpacking pip
Downloading pip-1.3.1.tar.gz (247Kb): 247Kb downloaded
Running setup.py egg_info for package pip
warning: no files found matching '*.html' under directory 'docs'
warning: no previously-included files matching '*.txt' found under directory 'docs/_build'
no previously-included directories found matching 'docs/_build/_sources'
Installing collected packages: pip
Found existing installation: pip 0.8
Uninstalling pip:
Successfully uninstalled pip
Running setup.py install for pip
warning: no files found matching '*.html' under directory 'docs'
warning: no previously-included files matching '*.txt' found under directory 'docs/_build'
no previously-included directories found matching 'docs/_build/_sources'
Installing pip script to /usr/bin
Installing pip-2.6 script to /usr/bin
Successfully installed pip
Cleaning up...

pip-python install --upgrade pip

Downloading/unpacking pip

Downloading pip-1.3.1.tar.gz (247Kb): 247Kb downloaded

Running setup.py egg_info for package pip

warning: no files found matching '*.html' under directory 'docs'

warning: no previously-included files matching '*.txt' found under directory 'docs/_build'

no previously-included directories found matching 'docs/_build/_sources'

Installing collected packages: pip

Found existing installation: pip 0.8

Uninstalling pip:

Successfully uninstalled pip

Running setup.py install for pip

warning: no files found matching '*.html' under directory 'docs'

warning: no previously-included files matching '*.txt' found under directory 'docs/_build'

no previously-included directories found matching 'docs/_build/_sources'

Installing pip script to /usr/bin

Installing pip-2.6 script to /usr/bin

Successfully installed pip

Cleaning up...

Читать далее Установка Django в виртуальное окружение на сервере с CentOS

django virtualenvs nginx uwsgi supervisor debian

Предполагается, что у нас уже есть настроенное виртуальное окружение virtualenv debian

nginx в стандартном репозитории Debian Squeeze старый, поэтому ставить будем из backports

/etc/apt/sources.list

deb http://backports.debian.org/debian-backports squeeze-backports main contrib non-free
deb http://ftp.ru.debian.org/debian testing main non-free contrib
deb http://ftp.ru.debian.org/debian unstable main non-free contrib

deb http://backports.debian.org/debian-backports squeeze-backports main contrib non-free

deb http://ftp.ru.debian.org/debian testing main non-free contrib

deb http://ftp.ru.debian.org/debian unstable main non-free contrib

/etc/apt/preferences

Package: *
Pin: release a=stable
Pin-Priority: 700
Package: *
Pin: release a=squeeze-backports
Pin-Priority: 675
Package: *
Pin: release a=testing
Pin-Priority: 650
Package: *
Pin: release a=unstable
Pin-Priority: 600

Package: *

Pin: release a=stable

Pin-Priority: 700

Package: *

Pin: release a=squeeze-backports

Pin-Priority: 675

Package: *

Pin: release a=testing

Pin-Priority: 650

Package: *

Pin: release a=unstable

Pin-Priority: 600

Указываем использовать stable по умолчанию

echo 'APT::Default-Release "stable";' > /etc/apt/apt.conf.d/default

1	echo 'APT::Default-Release "stable";' > /etc/apt/apt.conf.d/default

Обновляем список репозиториев

aptitude update

1	aptitude update

Читать далее django virtualenvs nginx uwsgi supervisor debian