显示标签为“textsearch”的博文。显示所有博文
显示标签为“textsearch”的博文。显示所有博文

2012年3月29日星期四

Full Text shutdown status

I have a large full text index (30 millions records) that is
populating on a live server (they are just doing without the full text
search stuff for now) and the index goes into shutdown mode. What is
that? Last time it did this i stopped and started the mssearch
service and it started back up again. The only problem is that when
it completed last time the number of records in the index was like 8
million. I did a test and a bunch of the rows did not get into the
population even though it supposedly finished correctly. So I am
hesitant to restart the service since I'm afriad it will not
completely fill the index again.
This is SQL 2000 on a Windows 2003 entprise box. But not in a
cluster. It's a stand alone machine. 4 proc, 16 gigs on the machine.
13 of the gigs used by sql. The box does stay pretty busy but not
completely bogged down.
Ryan
are there any event messages in the application log from MSSearch or MSSCI?
Hilary Cotter
Looking for a book on SQL Server replication?
http://www.nwsu.com/0974973602.html
"RyanOly" <ryanoly@.gmail.com> wrote in message
news:661760f9.0407141123.66388c39@.posting.google.c om...
> I have a large full text index (30 millions records) that is
> populating on a live server (they are just doing without the full text
> search stuff for now) and the index goes into shutdown mode. What is
> that? Last time it did this i stopped and started the mssearch
> service and it started back up again. The only problem is that when
> it completed last time the number of records in the index was like 8
> million. I did a test and a bunch of the rows did not get into the
> population even though it supposedly finished correctly. So I am
> hesitant to restart the service since I'm afriad it will not
> completely fill the index again.
> This is SQL 2000 on a Windows 2003 entprise box. But not in a
> cluster. It's a stand alone machine. 4 proc, 16 gigs on the machine.
> 13 of the gigs used by sql. The box does stay pretty busy but not
> completely bogged down.
> Ryan

2012年3月27日星期二

Full Text Search service

Hi,
anyone can tell me if whith Msde 2000a (Sp3a) or with Sp4 the full text
search service is supported?
Thanks in advance.
hi,
fc3 wrote:
> Hi,
> anyone can tell me if whith Msde 2000a (Sp3a) or with Sp4 the full
> text search service is supported?
> Thanks in advance.
nope... unfortunately MSDE does not support FullText service
Andrea Montanari (Microsoft MVP - SQL Server)
http://www.asql.biz/DbaMgr.shtmhttp://italy.mvps.org
DbaMgr2k ver 0.12.0 - DbaMgr ver 0.58.0
(my vb6+sql-dmo little try to provide MS MSDE 1.0 and MSDE 2000 a visual
interface)
-- remove DMO to reply
|||Grazie.
Many thanks.
"Andrea Montanari" wrote:

> hi,
> fc3 wrote:
> nope... unfortunately MSDE does not support FullText service
> --
> Andrea Montanari (Microsoft MVP - SQL Server)
> http://www.asql.biz/DbaMgr.shtmhttp://italy.mvps.org
> DbaMgr2k ver 0.12.0 - DbaMgr ver 0.58.0
> (my vb6+sql-dmo little try to provide MS MSDE 1.0 and MSDE 2000 a visual
> interface)
> -- remove DMO to reply
>
>
|||fc3 wrote:
> Grazie.
> Many thanks.
:D
prego
you are welcome
Andrea Montanari (Microsoft MVP - SQL Server)
http://www.asql.biz/DbaMgr.shtmhttp://italy.mvps.org
DbaMgr2k ver 0.12.0 - DbaMgr ver 0.58.0
(my vb6+sql-dmo little try to provide MS MSDE 1.0 and MSDE 2000 a visual
interface)
-- remove DMO to reply

2012年3月26日星期一

full text search on doc files

I have a list of word files (.doc) I am trying to us the full text
search in sql server to find key words in these word documents. I have
a Windows XP and sql server installed on my machine. ( no network
server). I cant seem to get the code to work to do a search on these
files. I would really appreciate if someone can send me some code from
start to finsh so I can create the sql tables and then catolog to
search for keywords in the word files.What version of SQL Server are you using? Word documents must be stored in
columns of data type image, and another column must be present to store the
type suffix (".doc" for MS Word files).
When full-text indexing is activated for an image column the name of the
column where the suffix is stored must also be set. How this is actually don
e
depends on the SQL Server version.
SQL 2000:
sp_fulltext_column
@.tabname = '<qualified_table_name>'
,@.colname = '<column_name>'
,@.action = 'add'
,@.type_colname = '<type_column_name>'
SQL 2005:
create fulltext index on <table_name>
(<column_name> TYPE COLUMN <type_column_name>
key index <index_name>
If the index has been created appropriately, then you might also check the
SQL Error Log:
http://milambda.blogspot.com/2005/1...n-or-bybug.html
ML
http://milambda.blogspot.com/|||ML,
How do i create the table first and get the files indexed? How do I
get the files catologued? i am using sql2000 . Do i run the sql you
wrote above after the files have been imported?|||Setting up full-text indexing in SQL 2000 is very well documented in Books
Online:
http://msdn.microsoft.com/library/d... />
d_6g1f.asp
After you've created the table and enabled the database for full-text
search, you need to enable the table, then the column for full-text indexing
.
It really doesn't matter whether you set up FTI before inserting data in the
tables, or after, as long as you propagate change to the catalogue before yo
u
try to use full-text search (new rows will not be found by the engine until
the contents have been indexed).
There are several methods of propagating change to the full-text catalogue:
full propagation, incremental propagation and background propagation (with
change tracking). With background propagation new rows are indexed after
they've been inserted and changed rows are indexed after they've been
updated, but there's a performance impact and writetext/updatetext operation
s
are not detected.
To avoid the downside you could use incremental propagation by creating a
SQL Agent Job that starts incremental propagation as frequently as you need.
Are you having any specific problems? You could also ask these questions in
the dedicated newsgorup: "microsoft.public.sqlserver.fulltext".
ML
http://milambda.blogspot.com/|||ML,
I have looked through the link, still not getting it to work. I guess
i need some code, from start to finish, loading the file from the
c:\... and then running the sql. Anything you may have would be
extremely helpful..thanks|||The link I sent you will help you set up full-text indexing, but the actual
method of inserting MS Word documents into the table depends on your client
application.
If you need to insert blobs through T-SQL, you could use BULK INSERT:
http://msdn.microsoft.com/library/d...br />
4fec.asp
Also make sure you create the format file in accordance to this KB article:
http://support.microsoft.com/defaul...kb;en-us;271344
Information on using format files:
http://msdn.microsoft.com/library/d...>
bcp_9yat.asp
ML
http://milambda.blogspot.com/

Full text search of varbinary column

I'm looking for suggestions on the best approach for creating a full text
search index on a varbinary column. The varbinary data is the binary
serialization of a custom clr datatype (but to be clear -- the column is
typed varbinary, not the custom type).
I thought I might be able to create a computed column that converted the
varbinary to the custom type, then used methods on the custom type to convert
to string. But it seems like I can't do that without persisting the computed
column which I really don't want to do.
Another approach I'm considering is to create an iFilter for the datatype,
then create a computed column that just returns the column type I'd register
for the iFilter. That just seems a little heavy weight to me and has
deployment headaches.
I can't help think I'm missing something obvious. Any ideas?
Thanks.
You could create the ifilter and have it emit the text data which is stored
in the varbinary column. I think you will find the overhead of this to be
significant, and would advise you to store the data as text data in a
varchar column and index that column.
http://www.zetainteractive.com - Shift Happens!
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Geoff Chappell" <GeoffChappell@.discussions.microsoft.com> wrote in message
news:42CBA330-A7CC-4E7C-BEF9-663E96BA3A69@.microsoft.com...
> I'm looking for suggestions on the best approach for creating a full text
> search index on a varbinary column. The varbinary data is the binary
> serialization of a custom clr datatype (but to be clear -- the column is
> typed varbinary, not the custom type).
> I thought I might be able to create a computed column that converted the
> varbinary to the custom type, then used methods on the custom type to
> convert
> to string. But it seems like I can't do that without persisting the
> computed
> column which I really don't want to do.
> Another approach I'm considering is to create an iFilter for the datatype,
> then create a computed column that just returns the column type I'd
> register
> for the iFilter. That just seems a little heavy weight to me and has
> deployment headaches.
> I can't help think I'm missing something obvious. Any ideas?
> Thanks.
|||Thanks. I'm leaning towards the approach you suggest.
It seems like it comes down to a space/time tradeoff to some extent -- I can
either take the hit of the space required to materialize an alternate
representation (e.g. persisted computed column, indexed view, etc.), or pay
the runtime costs of an ifilter (there's also a time cost to any
materialization, and a complexity cost to the ifilter so it's not quite that
simple).
One of the things that bugs me about the ifilter approach is that it seems
like a hack since I have to fake a document extension for my custom datatype
to make it work. Do you know if any other values are accepted for Column
Type? e.g. can I just use the clsid of the filter?
-Geoff
"Hilary Cotter" wrote:

> You could create the ifilter and have it emit the text data which is stored
> in the varbinary column. I think you will find the overhead of this to be
> significant, and would advise you to store the data as text data in a
> varchar column and index that column.
> --
> http://www.zetainteractive.com - Shift Happens!
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTS
> http://www.indexserverfaq.com
> "Geoff Chappell" <GeoffChappell@.discussions.microsoft.com> wrote in message
> news:42CBA330-A7CC-4E7C-BEF9-663E96BA3A69@.microsoft.com...
>
>

Full text search of varbinary column

I'm looking for suggestions on the best approach for creating a full text
search index on a varbinary column. The varbinary data is the binary
serialization of a custom clr datatype (but to be clear -- the column is
typed varbinary, not the custom type).
I thought I might be able to create a computed column that converted the
varbinary to the custom type, then used methods on the custom type to conver
t
to string. But it seems like I can't do that without persisting the computed
column which I really don't want to do.
Another approach I'm considering is to create an iFilter for the datatype,
then create a computed column that just returns the column type I'd register
for the iFilter. That just seems a little heavy weight to me and has
deployment headaches.
I can't help think I'm missing something obvious. Any ideas?
Thanks.You could create the ifilter and have it emit the text data which is stored
in the varbinary column. I think you will find the overhead of this to be
significant, and would advise you to store the data as text data in a
varchar column and index that column.
http://www.zetainteractive.com - Shift Happens!
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"Geoff Chappell" <GeoffChappell@.discussions.microsoft.com> wrote in message
news:42CBA330-A7CC-4E7C-BEF9-663E96BA3A69@.microsoft.com...
> I'm looking for suggestions on the best approach for creating a full text
> search index on a varbinary column. The varbinary data is the binary
> serialization of a custom clr datatype (but to be clear -- the column is
> typed varbinary, not the custom type).
> I thought I might be able to create a computed column that converted the
> varbinary to the custom type, then used methods on the custom type to
> convert
> to string. But it seems like I can't do that without persisting the
> computed
> column which I really don't want to do.
> Another approach I'm considering is to create an iFilter for the datatype,
> then create a computed column that just returns the column type I'd
> register
> for the iFilter. That just seems a little heavy weight to me and has
> deployment headaches.
> I can't help think I'm missing something obvious. Any ideas?
> Thanks.|||Thanks. I'm leaning towards the approach you suggest.
It seems like it comes down to a space/time tradeoff to some extent -- I can
either take the hit of the space required to materialize an alternate
representation (e.g. persisted computed column, indexed view, etc.), or pay
the runtime costs of an ifilter (there's also a time cost to any
materialization, and a complexity cost to the ifilter so it's not quite that
simple).
One of the things that bugs me about the ifilter approach is that it seems
like a hack since I have to fake a document extension for my custom datatype
to make it work. Do you know if any other values are accepted for Column
Type? e.g. can I just use the clsid of the filter?
-Geoff
"Hilary Cotter" wrote:

> You could create the ifilter and have it emit the text data which is store
d
> in the varbinary column. I think you will find the overhead of this to be
> significant, and would advise you to store the data as text data in a
> varchar column and index that column.
> --
> http://www.zetainteractive.com - Shift Happens!
> Looking for a SQL Server replication book?
> http://www.nwsu.com/0974973602.html
> Looking for a FAQ on Indexing Services/SQL FTS
> http://www.indexserverfaq.com
> "Geoff Chappell" <GeoffChappell@.discussions.microsoft.com> wrote in messag
e
> news:42CBA330-A7CC-4E7C-BEF9-663E96BA3A69@.microsoft.com...
>
>

2012年3月22日星期四

Full Text Search Filters What is installed?


Is there a way to query and identify which file extensions have a Full Text
Search filter installed for SQL2005?
Thanks
TomLook up sys.fulltext_document_types catalog view in Books Online.
select *
from sys.fulltext_document_types
ML
http://milambda.blogspot.com/|||that only gives the iFilters which ship with SQL Server 2005. If you choose
to load the ones via
exec sp_fulltext_service 'load_os_resources', 1;
exec sp_fulltext_service 'verify_signature', 0;
go
you need to get a complete list via filtreg
Hilary Cotter
Director of Text Mining and Database Strategy
RelevantNOISE.Com - Dedicated to mining blogs for business intelligence.
This posting is my own and doesn't necessarily represent RelevantNoise's
positions, strategies or opinions.
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"ML" <ML@.discussions.microsoft.com> wrote in message
news:3388D19A-B26B-470A-9E08-497B93987A26@.microsoft.com...
> Look up sys.fulltext_document_types catalog view in Books Online.
> select *
> from sys.fulltext_document_types
>
> ML
> --
> http://milambda.blogspot.com/|||Good to know, thanks.
ML
http://milambda.blogspot.com/|||Hi ML,
I got the error when run the SQL:
Invalid object name 'sys.fulltext_document_types'
"ML" <ML@.discussions.microsoft.com> wrote in message
news:3A88C201-4D67-4540-A064-23B553D2D932@.microsoft.com...
> Good to know, thanks.
>
> ML
> --
> http://milambda.blogspot.com/|||Are you trying this on SQL 2005?
ML
http://milambda.blogspot.com/

full text search book

I would like to ask if there are any books for beginners on full text
search for sql server thanks!
I am unaware of one. You might want to check out SQL Server 2005 Bible, I
wrote a chapter on SQL FTS there.
Hilary Cotter
Looking for a SQL Server replication book?
http://www.nwsu.com/0974973602.html
Looking for a FAQ on Indexing Services/SQL FTS
http://www.indexserverfaq.com
"savvas stampolloglou" <savvito@.gmail.com> wrote in message
news:1169928806.822515.114140@.k78g2000cwa.googlegr oups.com...
>I would like to ask if there are any books for beginners on full text
> search for sql server thanks!
>