Pages tagged db:

ブラウザ上でさくさくデータベースの設計ができる『WWW SQL Designer』がすごすぎる - IDEA*IDEA ~ 百式管理人のライフハックブログ ~
http://www.ideaxidea.com/archives/2009/01/www_sql_designer.html

これは次回使いたい。
32 Tips To Speed Up Your MySQL Queries | AjaxLine
http://www.ajaxline.com/32-tips-to-speed-up-your-mysql-queries
Is the Relational Database Doomed? - ReadWriteWeb
http://www.readwriteweb.com/archives/is_the_relational_database_doomed.php
Recently, a lot of new non-relational databases have cropped up both inside and outside the cloud.
Article about where key/value databases should be used over relational databases, with some examples of dbs available.
purpose of the key/value databases. is the paradigm changing these days?
Tokyo Cabinet: Beyond Key-Value Store - igvita.com
http://www.igvita.com/2009/02/13/tokyo-cabinet-beyond-key-value-store/
SAVE N SHARE
database blog
A database lib
bdb alternative und sehr schnell
漢(オトコ)のコンピュータ道: MySQLを高速化する10の方法
http://nippondanji.blogspot.com/2009/02/mysql10.html
tuning tips
DBのチューニング
PostgreSQLを高速化する16のポイント
http://neta.ywcafe.net/000960.html
ここ数年のPostgreSQLの進化はすさまじく、2009年現在においてその性能はOracleと同等かそれより速い。
postgresqlを使う時に読み返したい
HowFriendFeedUsesMySqlToStoreSchemaLessData - FriendFeed では MySQL を使いどのようにスキーマレスのデータを保存しているのか
http://hyuki.com/yukiwiki/wiki.cgi?HowFriendFeedUsesMySqlToStoreSchemaLessData
昨日のの邦訳
redis - Google Code
http://code.google.com/p/redis/
Redis is a key-value database. It is similar to memcached but the dataset is not volatile, and values can be strings, exactly like in memcached, but also lists and sets with atomic operations to push/pop elements.
“Redis is a key-value database. It is similar to memcached but the dataset is not volatile, and values can be strings, exactly like in memcached, but also lists and sets with atomic operations to push/pop elements. “In order to be very fast but at the same time persistent the whole dataset is taken in memory and from time to time and/or when a number of changes to the dataset are performed it is written asynchronously on disk. You may lost the last few queries that is acceptable in many applications but it is as fast as an in memory DB (beta 6 of Redis includes initial support for master-slave replication in order to solve this problem by redundancy).”
A nice fast K/V data store, with some nice list/set features.
漢(オトコ)のコンピュータ道: さらにMySQLを高速化する7つの方法
http://nippondanji.blogspot.com/2009/03/mysql7.html
おまけ: Sharding
Web屋のネタ帳さんの記事では16のポイントが紹介されているが、漢(オトコ)のコンピュータ道の記事は10の方法だったのであと6つ足りない。オトコは数で勝負!!というわけで今日はネタを振り絞ってさらに7つのMySQL高速化テクニックを紹介しよう。
さらにMySQLを高速化する7つの方法
Test Center: Slacker databases break all the old rules | InfoWorld | Test Center | March 24, 2009 | By Peter Wayner
http://www.infoworld.com/archives/emailPrint.jsp?R=printThis&A=/article/09/03/24/12TC-databases_1.html
non-relation db comparision
JSONの可能性がグンと拡がるぞ! JSONスキーマ - 檜山正幸のキマイラ飼育記
http://d.hatena.ne.jp/m-hiyama/20090413/1239581682
JSONスキーマ
Persevere: The JSON database and JavaScript application server
http://www.persvr.org/
Jedi/Sector One's random thoughts - An overview of modern SQL-free databases
http://00f.net/2009/an-overview-of-modern-sql-free-databases
Backup your Database in Git | Viget Extend
http://www.viget.com/extend/backup-your-database-in-git/
When you think about it, a database dump is just SQL code, so why not manage it the same way you manage the rest of your code — in a source code manager? Setting such a scheme up is dead simple. On your production server, with git installed:
MIT Database Systems (6.830) TA Course Notes - marcua's blog
http://blog.marcua.net/post/117671929/mit-database-systems-6-830-ta-course-notes
Computer Science and Artificial Intelligence Laboratory. Navigate * Website * Twitter * Subscribe * Archives * Random Subscribe by Email MIT Database Systems (6.830) TA Course Notes In Fall 2008, I had the pleasure of TAing Database Systems with Sam Madden, Mike Stonebraker, and Evan Jones. I figured that I could take notes to help students follow the lectures while clarifying any confusing points that were raised during discussion. It would also help me avoid the embarrassment of forgetting something mentioned during a lecture and having students explain it to me during office hours:). I decided to take notes in plain text, mostly out of laziness. This turned out to be a challenge for drawing things like query plans, but forced me to distill explanations into a conversational tone that provided an alternative to traditional diagrams. Some students in the class told me that they benefited from and enjoyed the notes, and so I decided to open them up for reuse
Why CouchDB?
http://books.couchdb.org/relax/why-couchdb
man, I really really wish I understood this stuff.
shows
“Django may be built for the Web, but CouchDB is built of the Web. I’ve never seen software that so completely embraces the philosophies behind HTTP. CouchDB makes Django look old-school in the same way that Django makes ASP look outdated.”
ebook on why you would choose couchdb
CouchDB: Perform like a pr0n star
http://www.slideshare.net/mattetti/couchdb-perform-like-a-pr0n-star
Check out this SlideShare Presentation : CouchDB: Perform like a pr0n star http://tinyurl.com/cukfou [from http://twitter.com/josefrichter/statuses/1588959474]
SitePen Blog » JavaScriptDB: Persevere’s New High-Performance Storage Engine
http://www.sitepen.com/blog/2009/04/20/javascriptdb-perseveres-new-high-performance-storage-engine/
JavaScriptDB: Persevere’s New High-Performance Storage Engine April 20th, 2009 at 8:47 pm by Kris Zyp The latest beta of Persevere features a new native object storage engine called JavaScriptDB that provides high-end scalability and performance. Persevere now outperforms the common PHP and MySQL combination for accessing data via HTTP by about 40% and outperforms CouchDB by 249%. The new storage engine is designed and optimized specifically for persisting JavaScript and JSON data with dynamic object structures. It is also built for extreme scalability, with support for up to 9,000 petabytes of JSON/JS data in addition to any binary data.
Should you go Beyond Relational Databases? | Think Vitamin
http://thinkvitamin.com/dev/should-you-go-beyond-relational-databases/
Relational databases, such as MySQL, PostgreSQL and various commercial products, have served us well for many years. Lately, however, there has been a lot of discussion on whether the relational model is reaching the end of its life-span, and what may come after it.
Alternatives to SQL dbs - document, key-value, graph databases
braindump: NOSQL debrief
http://blog.oskarsson.nu/2009/06/nosql-debrief.html
NOSQL debrief
braindump: NOSQL debrief
First ever meeting of the NoSQL community. Lists all the presentations that were given.
Adding Simplicity - An Engineering Mantra: Shard Lessons
http://www.addsimplicity.com/adding_simplicity_an_engi/2008/08/shard-lessons.html
No, not SHARED lessons, I mean SHARD lessons. I have to admit that until about a year ago I didn't really know the term shards in relation to databases. Now don't confuse that with not understanding how databases can be horizontally scaled. I was introduced to that concept and helped to define the various ways it can be done but we just called it splits. Regardless of what you call it, there are some interesting challenges that are introduced. The well known challenges of consistency are discussed ad nauseam, even by me, so I'm not going there with this article. But besides that, there are some other lessons to learn when applying the pattern to your data.
Worth reading just for the section on intelligently designing shard counts. Great discussion on picking counts that smooth your cost step function
How b-tree database indexes work and how to tell if they are efficient (100' level) | mattfleming.com
http://mattfleming.com/node/192
A team member thought we should add an index on a 90 million row table to improve performance. The field on which he wanted to create this index had only four possible values. To which I replied that an index on a low cardinality field wasn't really going to help anything. My boss then asked me why wouldn't it help? I sputtered around for a response but ended up telling him that I'd get back to him with a reasonable explanation.
Imported from http://twitter.com/newsycombinator/status/2645303258 How b-tree database indexes work and how to tell if they are efficient http://bit.ly/dd6mf
HadoopDB Project
http://db.cs.yale.edu/hadoopdb/hadoopdb.html
An Architectural Hybrid of MapReduce and DBMS Technologies for Analytical Workloads.
HadoopDB is: 1. A hybrid of DBMS and MapReduce technologies that targets analytical workloads 2. Designed to run on a shared-nothing cluster of commodity machines, or in the cloud 3. An attempt to fill the gap in the market for a free and open source parallel DBMS 4. Much more scalable than currently available parallel database systems and DBMS/MapReduce hybrid systems. 5. As scalable as Hadoop, while achieving superior performance on structured data analysis workloads
Choosing a non-relational database; why we migrated from MySQL to MongoDB « Boxed Ice Blog
http://blog.boxedice.com/2009/07/25/choosing-a-non-relational-database-why-we-migrated-from-mysql-to-mongodb/
RethinkDB - The database for solid state drives.
http://www.rethinkdb.com/
Adminer
http://www.adminer.org/en/
Single-file PHP MySQL database administration tool
Adminer (formerly phpMinAdmin) is a full-featured MySQL management tool written in PHP. Conversely to phpMyAdmin, it consist of a single file ready to deploy to the target server.
an alternative to phpMyAdmin
database_software [Internet Mindmap]
http://internetmindmap.com/database_software
Good list
list of many database softwares and categorized
Liste de logiciels de bases de données classés par modèle de gestion de l'information (relationnel, XML, RDF...)
bobby-tables.com: A guide to preventing SQL injection
http://bobby-tables.com/
漢(オトコ)のコンピュータ道: なぜMySQLのサブクエリは遅いのか。
http://nippondanji.blogspot.com/2009/03/mysql_25.html
サブクエリ、あんまりよくない理由
というわけでMySQLによるサブクエリの処理について見てきたが、きちんと気をつけて使えばサブクエリも高速に実行される。もちろんJOINに書き換えた方が速いのは言うまでもないが、SQL文のメンテナンスし易さなどを考えるとサブクエリで処理を書きたい!という人も居るのではないだろうか。そんな方は次の事に気をつけてサブクエリを使って頂きたい。 * サブクエリの種類 * 外部クエリとサブクエリの評価の順序 * 外部クエリにおいてフェッチされる行数 * サブクエリで利用されるインデックス * テンポラリテーブルのサイズ
Amazon Web Services Blog: Don't Forget: You Can Use Amazon SimpleDB For Free!
http://aws.typepad.com/aws/2009/10/dont-forget-you-can-use-amazon-simpledb-for-free.html
PostgreSQL Tips and Tricks | gtuhl: startup technology
http://blog.gtuhl.com/2009/08/07/postgresql-tips-and-tricks/
Here’s a dozen tips for working with a PostgreSQL database. It is a sophisticated and powerful piece of software and just knowing a few rules of thumb before diving in can be a huge help.
Here’s a dozen tips for working with a PostgreSQL database. It is a sophisticated and powerful piece of software and just knowing a few rules of thumb before diving in can be a huge help. If you want more detail read the amazing documention. My list of tips was very long so I just chopped off a dozen for this post.
The End of a DBMS Era (Might be Upon Us) | blog@CACM | Communications of the ACM
http://cacm.acm.org/blogs/blog-cacm/32212-the-end-of-a-dbms-era-might-be-upon-us/fulltext
"Relational database management systems (DBMSs) have been remarkably successful in capturing the DBMS marketplace. To a first approximation they are “the only game in town,” and the major vendors (IBM, Oracle, and Microsoft) enjoy an overwhelming market share. They are selling “one size fits all”; i.e., a single relational engine appropriate for all DBMS needs. Moreover, the code line from all of the major vendors is quite elderly, in all cases dating from the 1980s. Hence, the major vendors sell software that is a quarter century old, and has been extended and morphed to meet today’s needs. In my opinion, these legacy systems are at the end of their useful life. They deserve to be sent to the “home for tired software.” Here’s why."
NoSQL: Distributed and Scalable Non-Relational Database Systems | Linux Magazine
http://www.linux-mag.com/cache/7579/1.html
From @jesserobbins
Non-SQL oriented distributed databases are all the rage in some circles. They’re designed to scale from day 1 and offer reliability in the face of failures.
NoSQL: Distributed and Scalable Non-Relational Database Systems
l
Cassandra and Ruby: A Love Affair? | Engine Yard Blog
http://www.engineyard.com/blog/2009/cassandra-and-ruby-a-love-affair/
"Most of today’s up and coming key-value stores are more than just simple key-value stores. You saw this when we looked at Tokyo Cabinet which, in addition to simple key-value capabilities, adds more sophisticated abilities, such as database-like tables. In this post we’ll look at Cassandra — a modern key-value store that continues this trend. Cassandra was originally developed by Facebook and released to open source last year. The Facebook team describes Cassandra as (Google) BigTable running on top of an Amazon Dynamo-like infrastructure."
Most of today's and up and coming key-value stores are more than just simple key-value stores. Cassandra is a modern key-value store that continues this trend.
MongoDB: A Light in the Darkness! (Key Value Stores Part 5) | Engine Yard Blog
http://www.engineyard.com/blog/2009/mongodb-a-light-in-the-darkness-key-value-stores-part-5/
Really interesting article about mongoDB and about the installation procedure
"MongoDB can be thought of as the goodness that erupts when a traditional key-value store collides with a relational database management system, mixing their essences into something that’s not quite either, but rather something novel and fascinating. -- MongoDB support is available in many languages, making it a good choice for a system that has to work in a polyglot environment; all of the major languages have support."
レプリケーションしてるMySQLで、マスタやスレーブが障害停止した場合のリカバリプラン - (ひ)メモ
http://d.hatena.ne.jp/hirose31/20091023/1256259405
mysqlのレプリケーションを用いた構成について書いたblog記事。まだきちんと読んでいない。 フェールオーバー時のデータの完全性を保証しない構成。 想定する運用フローが書いてあるので、自分で考えるときの参考にする
Open database life: MyISAMとInnoDBのどちらを使うべきか
http://opendatabaselife.blogspot.com/2009/10/myisaminnodb.html
MySQLのInnnoDB、MyISAMの利点・欠点がまとめられている。参考になります。
自分は、特別な事情が無い限り、5.1最新版に含まれるInnoDB Pluginを勧めています。
fault-tolerance.png (PNG Image, 784x393 pixels)
http://browsertoolkit.com/fault-tolerance.png
(PNG-Grafik, 784x393 Pixel)
All I want for Christmas is a SQL database with no JOINs, secondary indexes, UNIONs, views, character sets or anything else. Just exact and range primary key lookups, GROUP BY, ORDER BY, LIMIT and SQL_CALC_FOUND_ROWS.
SQL Databases Don't Scale
http://adam.blog.heroku.com/past/2009/7/6/sql_databases_dont_scale/
"Sharding kills most of the value of a relational database."
sql database db
データベースの基礎を理解しよう! プログラミング未経験から始めるPHP入門:CodeZine
http://codezine.jp/article/detail/3685
実録、ほぼ無停止なMySQLのフェイルオーバ (動画もあるよ) - (ひ)メモ
http://d.hatena.ne.jp/hirose31/20091111/1257942168
keepalived --vrrp で、マルチマスターフェイルオーバーする
Lawnchair
http://brianleroux.github.com/lawnchair/
A client side JSON document store. Want to see this in Node.JS.
Sorta like a couch except smaller and outside, also, a client side JSON document store. Perfect for webkit mobile apps that need a lightweight, simple and elegant persistence solution.
"Sorta like a couch except smaller and outside, also, a client side JSON document store. Perfect for webkit mobile apps that need a lightweight, simple and elegant persistence solution."
mysql と drizzle の負荷テストツール「skyload」が凄い! - id:kazuhookuのメモ置き場
http://d.hatena.ne.jp/kazuhooku/20090707/1246950315
MySQLの負荷テストに便利らしい。
Introducing Redis: a fast key-value database | Zen and the Art of Programming
http://antoniocangiano.com/2009/03/11/introducing-redis-a-key-value-database/
漢(オトコ)のコンピュータ道: MySQLレプリケーションを安全に利用するための10のテクニック
http://nippondanji.blogspot.com/2009/03/mysql10.html
2年に1回起きる可能性があるわけで、確かに何か対策した方がよいですね。 しかしケーブルや機材を取り替えて推奨値以上の環境を用意すれば2000万年に1回。10GbEthernetでも200万年に1回の確率ですね・・・それでも対策しないよりも対策した方がよいですが・・・ Okunoさんはバイナリログがネットワークによって化けた経験がおありなんですよね?
"1. マルチマスターレプリケーションを利用しない 非常によくある誤解なのだが、HAにしたいからといってマルチマスター構成にしているユーザをたまに見かける。マルチマスターとは2台のMySQLサーバで構成するトポロジのことで、2つのサーバが互いに相手のマスターかつスレーブとなりレプリケーションを行う。マルチマスターは両方のホストで更新が可能なのだが、片方のサーバ行われた更新は非同期でもう一方のサーバへ適用されるため、更新を行っている方のサーバがクラッシュした場合には更新が失われる可能性がある。"
クックパッドとHadoop « クックパッド開発者ブログ
http://techlife.cookpad.com/2009/09/16/cookpad-hadoop-introduction/
わかりやすい資料。
クックパッドとHadoop « クックパッド開発者ブログ はじめまして。今年の5月に入社した勝間@さがすチームです。入社してからは、なかなか大変なことも多いですが、最近はお酒好きが集まって月曜から飲み合う 「勝間会」なるものも発足して、仕事面でも仕事以外の面でも密度の高い毎日を過ごしています!さて、僕は「さがす」チーム所属ということで、普段はレシピを「さがす」ユーザの満足度を上げるために、 クックパッドの検索まわりについて、いろいろな開発を行っていま... はてなブックマーク - クックパッドとHadoop « クックパッド開発者ブログ はてなブックマークに追加 dann dann hadoop
Harish Mallipeddi's Blog - CouchDB naked
http://blog.poundbang.in/post/132952897/couchdb-naked
Good explanation of how CouchDB indexes.
how couchdb b-trees work internally
Why I think Mongo is to Databases what Rails was to Frameworks // RailsTips by John Nunemaker
http://railstips.org/2009/12/18/why-i-think-mongo-is-to-databases-what-rails-was-to-frameworks
Below are 7 Mongo and MongoMapper related features that I have found to be really awesome while working on switching Harmony, a new website management system by my company, Ordered List, to Mongo from MySQL.
The more I work with Mongo the more I am coming around to this way of thinking. I tell no lie when I say that I now approach Mongo with the same kind of excitement I first felt using Rails. For some, that may be enough, but for others, you probably require more than a feeling to check out a new technology
Yahoo!オークションでのMySQL 冗長化技術 (Yahoo! JAPAN Tech Blog)
http://techblog.yahoo.co.jp/cat207/db/mysql_failover/
よくある構成っちゃ、よくある構成。MySQL ClusterとかMySQL Proxyとかの完成度があがっていくとこの部分の構成がもっとシンプルかつ強いシステムになっていくのかも
「dual master + ソフトウエアロードバランサ + Nagios」「masterサーバーに障害があった場合には、(Nagios のイベントハンドラで)それに直結する slave のヘルスチェック用ポートを閉じる」
事例
dual master, ソフトウェアロードバランサ, 仮想DNS, gethostbyname書き換え, masterが片方落ちたら直下のslaveもnagiosのイベントハンドラで落として一貫性保つ
もう1つの、DBのかたち、分散Key-Valueストアとは (1/3) - @IT
http://www.atmarkit.co.jp/fjava/rensai4/bigtable01/01.html
キーバリューストアの解説 「CAP定理」では、分散システムで以下の3つを同時に保証することは不可能であることが示されています。 * データの整合性(Consistency) * データの可用性(Availability) * データの分散化(Partition-tolerance)
>RDBとは別の、クラウド時代のデータベースとして注目を浴びている「分散Key-Valueストア」。その本命ともいえる、Googleの数々のサービスの基盤技術「Bigtable」について徹底解説 どうかなあ…
Bigtable, SimpleDB, Tokyo Tyrant
Vineet Gupta: NoSql Databases – Part 1 - Landscape
http://www.vineetgupta.com/2010/01/nosql-databases-part-1-landscape.html
At Directi, we are taking a hard look at the way our applications need to store and retrieve data, and whether we really need to use a traditional RDBMS for all scenarios. This does not mean that we will eschew relational systems altogether. What it means is that we will use the best tool for the job – we will use non-relational options wherever needed and not throw everything at a relational database with a mindless one-size-fits-all approach. ... ... This post covers the current landscape of the NoSQL space. In a subsequent post, I intend to cover in more detail the various problem areas addressed by NoSQL systems and the specific algorithms used.
Really detailed description of a number of NoSQL solutions. Interesting reading on Cassandra and Voldemort.
This post covers the current landscape of the NoSQL space. In a subsequent post, I intend to cover in more detail the various problem areas addressed by NoSQL systems and the specific algorithms used.
FleetDB
http://fleetdb.org/
FleetDB is a schema-free database optimized for agile development.
ウノウラボ Unoh Labs: RDBで階層構造を扱うには?
http://labs.unoh.net/2009/06/rdb.html
how to store tree structure on RDB
階層構造
DB設計の神ツール「ERMaster」なら、ここまでできる (1/3) - @IT
http://www.atmarkit.co.jp/fjava/rensai4/devtool11/devtool11_1.html
ERMasterは、ほかのツールに比べ、直感的で分かりやすいUI(ユーザーインターフェイス)に、カスタマイズ可能な、Excelで出力できるテーブル定義書、辞書機能など痒いところに手が届くERモデリングのツールです。本稿では、このERMasterについてご紹介します。
ERMasterは、ほかのツールに比べ、直感的で分かりやすいUI(ユーザーインターフェイス)に、カスタマイズ可能な、Excelで出力できるテーブル定義書、辞書機能など痒いところに手が届くERモデリングのツールです
いくつかの無料で利用できるツールが提供されているので、筆者はそれらを利用していましたが、最近「ERMaster」と呼ばれるEclipseプラグインの存在を知りました。  ERMasterは、ほかのツールに比べ、直感的で分かりやすいUI(ユーザーインターフェイス)に、カスタマイズ可能な、Excelで出力できるテーブル定義書、辞書機能など痒いところに手が届くERモデリングのツールです。本稿では、このERMasterについてご紹介します。
分散Key-Valueストア「kumofs」を公開しました! - 古橋貞之の日記
http://d.hatena.ne.jp/viver/20100118/p1
データベースを用いたセッションデータ管理について - Slow Dance
http://d.hatena.ne.jp/LukeSilvia/20090523/p1
InnoDBは行ロックに対応。6000万レコード。。。
Web アプリケーションとは切っても切れないセッション機構。DB ベースでセッション管理を行なって得られた知見と、それを元に考察した結果をまとめてみます。
mixi Engineers’ Blog » PerlとRubyで省メモリなハッシュを使おう
http://alpha.mixi.co.jp/blog/?p=791
Leo's Chronicle: ぜひ押さえておきたいデータベースの教科書
http://leoclock.blogspot.com/2009/01/blog-post_07.html
Notes from a production MongoDB deployment « Boxed Ice Blog
http://blog.boxedice.com/2010/02/28/notes-from-a-production-mongodb-deployment/
Mongo DB Production
Interesting blog post detailing production experiences with mongodb.
Dennis Forbes on Software and Technology - Getting Real about NoSQL and the SQL-Isn't-Scalable Lie
http://www.yafla.com/dforbes/Getting_Real_about_NoSQL_and_the_SQL_Isnt_Scalable_Lie/
SQL is Scalable and NoSQL Isn’t For Everyone The point is one that I think all rational people already realize: The ACID RDBMS isn’t appropriate for every need, nor is the NoSQL solution.
"[Though as Michael Stonebraker points out, SQL the query language actually has remarkably little to actually to do with the debate. It would be more clearly called NoACID]"
HyperGraphDB - A Graph Database
http://www.kobrix.com/hgdb.jsp
HyperGraphDB is a general purpose, extensible, portable, distributed, embeddable, open-source data storage mechanism. It is a graph database designed specifically for artificial intelligence and semantic web projects, it can also be used as an embedded object-oriented database for projects of all sizes.
Key-Value Store勉強会に行ってきました - blog.katsuma.tv
http://blog.katsuma.tv/2009/02/key_value_store_study.html
"# LuxIO (ラックスIO)"# 普通のB+-tree # 特徴1 * mapped index * index部を全部mmap o index部を実メモリより小さいシステムが対象 # 特徴2 * 長いvalue * 4Gまで * node size(page size)をこえたvalueも余計なオーバーヘッドなしで扱える # 特徴3 * 効率的なappend * paddingなしでLinkedListのデータ構造 # SSDに向いてる? # 使い道 * key-valともに小さいデータで構想なアクセスが必要な場合 * 実メモリ以下のデータベースという制約あり * 大きなvalueを扱いたい場合 * 大きなvalueをどんどん追記したい # 向かない処理 * 削除が多い処理 * 小さいデータをたくさんリンク o seekのオーバーヘッドが大きすぎる * Read,Writeの激しいアプリ # 分散はたぶんしない # Hashはつくるかも # read lockはなくしたい * 読み込みを重きをおく"
Key-Value型データ設計に関して。いくつかのシステムの特徴などのメモ。
10 sql tips to speed up your database
http://www.catswhocode.com/blog/10-sql-tips-to-speed-up-your-database
SQL optimization
10 sql tips to speed up your database http://bit.ly/9uIi6k #sql
CouchDB with CouchRest in 5 minutes « The Merbist
http://merbist.com/2009/05/17/couchdb-with-couchrest-in-5-minutes/
The other night, during our monthly SDRuby meetup, lots of people were very interested in learning more about CouchDB and Ruby. I tried to show what Couch was all about but I didn’t have time to show how to use CouchDB with Ruby. Here is me trying to do that in 10 minutes or less. I’ll assume you don’t have CouchDB installed.
CouchDB with CouchRest in 5 minutes The other night, during our monthly SDRuby meetup, lots of people were very interested in learning more about CouchDB and Ruby. I tried to show what Couch was all about but I didn’t have time to show how to use CouchDB with Ruby. Here is me trying to do that in 10 minutes or less. I’ll assume you don’t have CouchDB installed.
データベースパフォーマンスに関する、僕が知りうる限り最高の教科書 - 山本大@クロノスの日記
http://d.hatena.ne.jp/iad_otomamay/20090805/1249479181
データベースパフォーマンスアップの教科書 基本原理編
どこの現場に行っても正解を導く方程式は一緒なので応用が利く Oracle、SQLServer、MySQLと色々なDBのチューニングをしてきましたが、 どれもRDBの理論に基いているので基本原理を知ればチューニングは可能なはず インデックススキャンの種類や、実行計画の読み方もわかりやすく詳しい
Are Commercial Databases Worth It? - Coding the Wheel
http://www.codingthewheel.com/archives/are-commercial-databases-worth-it
I've worked with expensive SQL Server and Oracle setups for most of my career. I've defended them viciously against all comers and contrarians. I've participated in late-night guerilla flame wars and drunken bar brawls. And I've sought out with relentless tunnel vision those pieces of propaganda which support my foregone conclusion: that SQL Server and/or Oracle are (or were) the best choices for the organization. I used to be a commercial database advocate. These databases have put food on my table for a dozen years, you see. I am (or was) what you might call an entrenched practicioner, not necessarily an expert, but a practicioner. And in the manner of entrenched practicioners around the world, I've treated you heretics with the sadistic undercutting and poisonous rancor you've deserved! "MySQL?" I would sneer. "PostgreSQL? Thanks, but this a serious project. We need a database we can depend on." Ahem.
googled "why pay for commercial database" and found this among other articles
Are Commercial Databases Worth It? http://bit.ly/Du96H (via @newsycombinator) [from http://twitter.com/tadej/statuses/1664387681]
Se cuestiona la idoneidad de escoger una BD comercial com Oracle o SQL Server frente a sus alternativas Open Source.
Visual Guide to NoSQL Systems - Nathan Hurst's Blog
http://blog.nahurst.com/visual-guide-to-nosql-systems
Good discussion in the comments as well.
mixi Engineers’ Blog » 3行でできる超お手軽全文検索
http://alpha.mixi.co.jp/blog/?p=1112
タグ検索と全文検索といえば、Tokyo Dystopiaが同じような機能を既に実現しています。TCにタグ検索と全文検索がサポートされたからもうTDは不要なのかと思われるかもしれませんが、そうではありません。転置インデックスのライブラリとしてはTDの方がはるかに効率的かつスケールする設計になっていて、また業務に必要なカスタマイズを容易にするためにシンプルな実装になっています。一方でTCの転置インデックスは、パフォーマンスやスケーラビリティではTDに劣りますが、ものすごく簡単に導入できることが特徴です。既にテーブルDBでデータの管理をしているならば、setindexホゲホゲという文を書くだけで1分以内に検索機能を強化することができるのです
nothingmuch's most awesome Perl blog EVAR!!1one: Why I don't use CouchDB
http://blog.woobling.org/2009/05/why-i-dont-use-couchdb.html
Keep this as a reference to common couch FUD :)
Kazuho@Cybozu Labs: Pacific という名前の分散ストレージを作り始めた件
http://developer.cybozu.co.jp/kazuho/2009/06/pacific-18c7.html
Redis tutorial, April 2010 - by Simon Willison
http://simonwillison.net/static/2010/redis-tutorial/
posted by thraxil: http://quimby.ccnmtl.columbia.edu/ircbot/web/?y=2010&m=04&d=26#20100426105402
Awesome tutorial.
These slides and notes were originally written to accompany a three hour Redis tutorial I gave at the NoSQL Europe conference on the 22nd of April 2010.
mixi Engineers’ Blog » DBMによるテーブルデータベース
http://alpha.mixi.co.jp/blog/?p=290
テーブルデータベースとは 簡単に言えば、リレーショナルデータベースのテーブルのように、複数の列からなるレコードを格納できるデータベースです。SQLや表結合などの複雑な機能はサポートしませんが、そのぶん高速に動作します。つまり、DBMの速度で動くリレーショナル風データベースです(厳密にはリレーショナルデータベースではありません)。
Tokyo Cabinet DBM の使い方に関するチュートリアル
またひとつbigtableに近づいた感じ。mikioプロダクトはどのバージョンをどうやって入れて運用するのが楽なのかよく分からんところか。
table database
Readings in Database Systems Web Supplement
http://redbook.cs.berkeley.edu/
This book is one of the fundamental database theory books available today. A list of the papers featured in the book, as well as various lecture notes, are listed. Need to track down some of these papers.
無いから作った人たち:ITpro
http://itpro.nikkeibp.co.jp/article/OPINION/20090216/324752/
"memcachedの特徴は、データをキャッシュするメモリーとして、通常のPCサーバーの物理メモリーを利用すること。大容量データを複数のPCサーバーのメモリーに分散しておくために、「キー・バリュー型データストア」と呼ぶ方法を採用している。データをいったん非正規化し、「キー」とそれに対応する「値(バリュー)」にしてから保存する。データをキーと値の組み合わせにすることで、複数のサーバーに分散しておける。"
MySQLによるデータウェアハウス構築 (Yahoo! JAPAN Tech Blog)
http://techblog.yahoo.co.jp/web/yahoo/mysql/
バッチ
「キー・バリュー型データストア」開発者が大集合した夜:ITpro
http://itpro.nikkeibp.co.jp/article/OPINION/20090226/325527/
記者にとって驚きだったのは、現在日本で開発されているキー・バリュー型データストアがこの3つに留まらないことだった。しかも開発者は総じて若い。勉強会に参加する80人近くの技術者も、ほぼ同年代だった。
キー・バリュー型データストア(またはキー・バリュー型データベース)は、大量のユーザーとデータを抱え、データベースのパフォーマンス問題とコスト高に頭を悩ませるWeb企業が注目する技術である。
mmalone's django-caching at master - GitHub
http://github.com/mmalone/django-caching/tree/master
"Mike Malone shares code used by Pownce to add QuerySet level caching to Django. It’s a smart implementation—a CachingQuerySet class inspects the arguments passed to get(), and if they’re just a straight forward exact PK lookup hits memcache for the object before hitting the database. Signals are used to invalidate the cache."
Some examples of transparently caching things in Django. An example Django app that uses custom managers, fields, and QuerySets to transparently cache objects.
Some examples of transparently caching things in Django.
mmalone's django-caching app
VoltDB: Fast, Scalable SQL RDBMS with ACID
http://www.voltdb.com/
SCALABLE, OPEN-SOURCE SQL DBMS WITH ACID
怪異・妖怪画像データベース
http://www.nichibun.ac.jp/YoukaiGazouMenu/
Base de données des monstres de la mythologie japonaise.
怪異・妖怪画像データベース 怪異・妖怪画像データベース Copyright (c)2010- International Research Center for Japanese Studies, Kyoto, Japan. All rights reserved. はてなブックマーク - 怪異・妖怪画像データベース はてなブックマークに追加 gin-oi2 gin-oi2 データベース, **お役立ち, *webサービス いつか、役に立つときが来るかも…
Getting Started with HTML5 Local Databases « Dark Crimson Blog
http://blog.darkcrimson.com/2010/05/local-databases/
HTML 5 local databases. Gives step by step instructions for setting one up!
Starting with Safari 4, iPhone/iPad OS3, Chrome 5, and Opera 10.5 (Desktop), HTML5 Local Databases are now supported. I’ve been reading about local databases for quite some time and decided to do a write up with some basic examples on how to get started.
14 Starting with Safari 4, iPhone/iPad OS3, Chrome 5, and Opera 10.5 (Desktop), HTML5 Local Databases are now supported. I’ve been reading about local databases for quite some time and decided to do a write up with some basic examples on how to get started.
Membase.org
http://www.membase.org/
For those familiar with memcached, membase provides on-the-wire protocol compatibility, but adds disk persistence; hierarchical storage management; data replication; live cluster reconfiguration and rebalancing; and secure multi-tenancy with data partitioning. Like memcached, membase is simple, fast and elastic.
Persistent Key/Value Storage
Membase is an open-source (Apache 2.0 license) distributed, key-value database management system optimized for storing data behind interactive web applications. These applications must service many concurrent users; creating, storing, retrieving, aggregating, manipulating and presenting data in real-time. Supporting these requirements, membase processes data operations with quasi-deterministic low latency and high sustained throughput.
from oreilly news link
Membase is an open-source (Apache 2.0 license) distributed, key-value database management system optimized for storing data behind interactive web applications. These applications must service many concurrent users; creating, storing, retrieving, aggregating, manipulating and presenting data in real-time. Supporting these requirements, membase processes data operations with quasi-deterministic low latency and high sustained throughput. It scales linearly from a single-server deployment to a cluster of thousands of machines. And because membase does not require creation of a schema before storing data, it is a flexible, cost-effective place to Store Lots of Stuff.
Membase is an open-source (Apache 2.0 license) distributed, key-value database management system optimized for storing data behind interactive web applications. These applications must service many concurrent users; creating, storing, retrieving, aggregating, manipulating and presenting data in real-time. Supporting these requirements, membase processes data operations with quasi-deterministic low latency and high sustained throughput. It scales linearly from a single-server deployment to a cluster of thousands of machines. And because membase does not require creation of a schema before storing data, it is a flexible, cost-effective place to Store Lots of Stuff. The original membase source code was released as Open Source by NorthScale, Zynga and NHN to membase.org in June 2010.